Application of spectrogram fusion and deep learning for hand gesture classification based on forearm electromyographic signals

Abstract

Hand gesture recognition based on surface electromyographic (sEMG) signals plays a critical role in modern human–computer interaction systems, particularly in upperlimb prosthetic applications. This study presents a method for classifying six selected hand
gestures using sEMG signals acquired from three forearm muscles. The recorded signals were digitally filtered, and an automatic segmentation algorithm was developed to isolate individual gestures from the continuous muscle activity recordings. These segments were transformed into spectrograms using the short-time Fourier transform (STFT), which served as input data for various convolutional neural network (CNN) architectures. The study compares two approaches to data processing: one in which signals from each channel were analyzed separately, and another in which spectrograms from all three channels were fused into a single three channel input. The primary objective was to investigate which method better captures the inter-relationships between the activity patterns of different muscles. The models were trained and evaluated using cross-validation. The best-performing architecture achieved an accuracy of 99%. The results indicate that fusing spectrograms from multiple channels into a single input can enhance the classification performance of complex muscle activity patterns, particularly when the amount of available training data is limited.