Towards hand gesture recognition using a channel-wise cumulative spike train image-driven model

A research paper by scientists at Shanghai Jiao Tong University presented a novel channel-wise cumulative spike train image-driven model (cwCST-CNN) for hand gesture recognition.

The research paper, published on Mar. 21, 2025 in the journal Cyborg and Bionic Systems, leverage a custom convolutional neural network (CNN) to extract both local and global features for classifying hand gestures, by decomposing high-density surface EMG (HD-sEMG) signals into channel-wise cumulative spike trains (cw-CSTs) and reconstructing these into two-dimensional images based on the spatial layout of the electrodes.

Gesture recognition, as a natural human-computer interaction method, has broad application prospects in fields such as prosthetic control, rehabilitation training, and mixed reality. The commonly used surface electromyography (sEMG) signal analysis methods, such as those based on time-frequency domain features (such as RMS), often only capture rough neural control information, are susceptible to noise interference, and ignore the inherent spatial distribution characteristics of muscle movement. “With the development of high-density surface electromyography (HD sEMG) technology, the discharge sequences (Spike Trains) of motor units (MUs) obtained through decomposition can more directly reflect the neural system's control over muscles, providing more representative low dimensional neural control information for gesture recognition.” said the author Yang Yu, a researcher at Shanghai Jiao Tong University.

The research process of this paper is divided into the following steps: Firstly, the HD-sEMG electrode array is used to collect the electrical signals of the forearm muscles, and the signals are filtered, denoised, and abnormal channels are removed. Then, using an algorithm based on spatial propagation characteristics, the cumulative discharge sequence of each channel (cw-CST), is decomposed from the HD-sEMG signal to reflect the activity of adjacent motion units. Next, the cw-CST data of each channel is reconstructed into a two-dimensional image (cw-CST image) based on the spatial distribution of electrodes to capture the spatial activation patterns of neural control. Finally, a customized convolutional neural network is designed and trained for hand gesture recognition.

The experimental results show that the proposed cwCST-CNN method achieves a classification accuracy of 96.92 ± 1.77% in recognizing 10 gestures, which is significantly better than other comparative methods. Comparative analysis shows that features constructed based on cw-CST have better separability between different gestures (higher Separability Index) and consistency between training and testing data (lower Repeatability Index), providing strong support for improving recognition accuracy. “Our study provides a novel and effective solution for high-precision gesture recognition, with the potential for widespread application in human-computer interaction fields such as prosthetic control and rehabilitation training.” said Yang Yu.

Authors of the paper include Yang Yu, Zeyu Zhou, Yang Xu, Chen Chen, Weichao Guo, and Xinjun Sheng.

This work was supported in part by the National Natural Science Foundation of China under Grants 52205025, 52175021, 52205024.

The paper, “Towards Hand Gesture Recognition Using a Channel-Wise Cumulative Spike Train Image-Driven Model” was published in the journal Cyborg and Bionic Systems on Mar. 21, 2025, at DOI: 10.34133/cbsystems.0219.

END