A Contrastive Learning Network for Performance Metric and Assessment of Physical Rehabilitation Exercises
-
- Long Yao
- Xiamen Key Laboratory of Computer Vision and Pattern Recognition and the Department of Computer Science and Technology, Huaqiao University, Xiamen, China
-
- Qing Lei
- Xiamen Key Laboratory of Computer Vision and Pattern Recognition and the Department of Computer Science and Technology, Huaqiao University, Xiamen, China
-
- Hongbo Zhang
- Fujian Key Laboratory of Big Data Intelligence and Security, Huaqiao University, Xiamen, China
-
- Jixiang Du
- Fujian Key Laboratory of Big Data Intelligence and Security, Huaqiao University, Xiamen, China
-
- Shangce Gao
- Faculty of Engineering, University of Toyama, Toyama-shi, Japan
書誌事項
- 公開日
- 2023
- 資源種別
- journal article
- 権利情報
-
- https://creativecommons.org/licenses/by/4.0/legalcode
- DOI
-
- 10.1109/tnsre.2023.3317411
- 公開者
- Institute of Electrical and Electronics Engineers (IEEE)
この論文をさがす
説明
Human activity analysis in the legal monitoring environment plays an important role in the physical rehabilitation field, as it helps patients with physical injuries improve their postoperative conditions and reduce their medical costs. Recently, several deep learning-based action quality assessment (AQA) frameworks have been proposed to evaluate physical rehabilitation exercises. However, most of them treat this problem as a simple regression task, which requires both the action instance and its score label as input. This approach is limited by the fact that the annotations in this field usually consist of healthy or unhealthy labels rather than quality scores provided by professional physicians. Additionally, most of these methods cannot provide informative feedback on a patient's motion defects, which weakens their practical application. To address these problems, we propose a multi-task contrastive learning framework to learn subtle and critical differences from skeleton sequences to deal with the performance metric and AQA problems of physical rehabilitation exercises. Specifically, we propose a performance metric network that takes triplets of training samples as input for score generation. For the AQA task, the same contrast learning strategy is used, but pairwise training samples are fed into the action quality assessment network for score prediction. Notably, we propose quantifying the deviation of the joint attention matrix between different skeleton sequences and introducing it into the loss function of our learning network. It is proven that considering both score prediction loss and joint attention deviation loss improves physical exercises AQA performance. Furthermore, it helps to obtain informative feedback for patients to improve their motion defects by visualizing the joint attention matrix's difference. The proposed method is verified on the UI-PRMD and KIMORE datasets. Experimental results show that the proposed method achieves state-of-the-art performance.
収録刊行物
-
- IEEE Transactions on Neural Systems and Rehabilitation Engineering
-
IEEE Transactions on Neural Systems and Rehabilitation Engineering 31 3790-3802, 2023
Institute of Electrical and Electronics Engineers (IEEE)
