Abstract: To enable automatic and accurate quantitative evaluation of hand function rehabilitation in stroke patients, this paper proposes a skeleton-based gesture recognition and evaluation method. First, the MediaPipe framework is used to extract hand keypoints and connect them to form a hand skeleton, converting traditional RGB video datasets into hand skeleton datasets. Then, a 3D convolutional neural network (C3D) model is employed to train on and recognize hand functional movements. Based on correct recognition, further evaluation is conducted using the dynamic time warping (DTW) algorithm. The DTW distance between the motions of the healthy and affected hands performing the same action is calculated, aligning both temporally and spatially to represent the similarity in action execution. Experiments establish optimal DTW thresholds for distinguishing different rehabilitation ratings for each action, which serve as the criteria for quantitative evaluation.Results show that using skeletal data instead of traditional video improves gesture recognition accuracy to 99.01% and reduces training time. With the DTW algorithm, automatic hand function rehabilitation assessment is achieved.

Key words: dynamic gesture recognition, 3D convolutional neural network (CNN), MediaPipe framework, dynamic time warping (DTW), hand function rehabilitation