Original scientific paper
https://doi.org/10.32985/ijeces.17.8.2
Multimodal Fusion for Emotion Classification based on GSR, BVP, and SKT
Adhitya Velip
; Department of Electronics and Telecommunication Engineering, Goa College of Engineering, Affiliated to Goa University, Ponda, India
*
Hassanali G. Virani
; Department of Electronics and Telecommunication Engineering, Goa College of Engineering, Affiliated to Goa University, Ponda, India
Amita U. Dessai
; Department of Electronics and Telecommunication Engineering, Goa College of Engineering, Affiliated to Goa University, Ponda, India
* Corresponding author.
Abstract
Emotion recognition in next-generation healthcare is revolutionizing patient care by fostering empathetic and personalized interactions. Advanced systems that analyse facial expressions, speech, and physiological signals enhance our understanding of emotional states, ultimately improving patient experiences. However, interpreting emotions through visible cues can often lead to inaccuracies. This underscores the importance of physiological emotion recognition, which relies on involuntary responses such as heart rate and skin conductance to provide a more accurate assessment of emotion. Such insights are crucial across various fields: in healthcare for better treatment plans, in human-computer interaction for empathetic user experiences, in robotics for more intuitive communications, and in market research for gauging consumer reactions and optimizing strategies. Overall, the integration of physiological emotion recognition has the potential to revolutionize our understanding of and interactions with emotional experiences across numerous disciplines. Galvanic Skin Response (GSR), Blood Volume Pulse (BVP), and Skin Temperature (SKT) are physiological signals used for emotion classification because they are not easily manipulated by individuals. Researchers employ machine learning techniques for emotion classification using GSR, BVP, and SKT; however, more accurate models are needed. This paper proposes a novel multimodal and unimodal emotion classification system that leverages Galvanic Skin Response (GSR), Blood Volume Pulse (BVP), and Skin Temperature (SKT) parameters. The model's performance is evaluated on the PHYMER dataset, which stands for Physiological Dataset for Multimodal Emotion Recognition with Personality as Context. This paper also leverages mutual information feature selection in conjunction with a Random Forest classifier, achieving accuracies of 97% and 96% for arousal and valence, respectively, when analyzing Galvanic Skin Response (GSR). In the case of Blood Volume Pulse (BVP), notable accuracy of 96% for arousal and 90% for valence was reported. For both arousal and valence, SKT achieved 97% accuracy. Additionally, feature-level fusion resulted in arousal and valence accuracies of 97% and 98%, respectively. More compelling are the results from decision-level fusion, which achieved accuracies of 98% for arousal and 99% for valence. These findings not only demonstrate the robustness of our approach but also highlight the great potential for progress in emotion classification.
Keywords
Multimodal fusion; GSR; BVP; SKT; Chatterjee’s correlation coefficient; Nu-SVM; PHYMER;
Hrčak ID:
350765
URI
Publication date:
4.9.2026.
Visits: 0 *