Smart City Gnosys
Smart city article details
| Title | Dream: Deep Learning-Based Recognition Of Emotions From Multiple Affective Modalities Using Consumer-Grade Body Sensors And Video Cameras |
|---|---|
| ID_Doc | 21000 |
| Authors | Sharma A.; Kumar A. |
| Year | 2024 |
| Published | IEEE Transactions on Consumer Electronics, 70, 1 |
| DOI | http://dx.doi.org/10.1109/TCE.2023.3325317 |
| Abstract | Ambient smart cities exist on the intersection of digital technology, disruptive innovation and urban environments that now essentially augment affect empathy and intelligent interfacing for human computer interactions (HCI). This research puts forward a deep learning approach, DREAM, for recognition of emotions using three affective modalities (audio, video, physiological) to develop an empathetic HCI system using consumer electronic IoT sensors and cameras. Convolution network is used to train for physiological signals. VGG and ResNet have been used to pre-train the models for emotion recognition from video and audio signals. DREAM is then fine-tuned on the publicly available K-EmoCon dataset to accurately recognize emotion for each subject. K-EmoCon is annotated by seven persons for five discrete emotions, and two affect dimensions. Finally, a probability-based average decision-level fusion strategy is used for combining the outputs of all the modalities. Leave one out strategy is used to train and evaluate the model for subject specific accuracies. For discrete emotions highest accuracy of 81.7% and 82.4% is achieved for dimensional emotions. DREAM has performed better than existing state-of-the-art for both emotion models. © 1975-2011 IEEE. |
| Author Keywords | Facial emotion recognition; human computer interaction; IoT sensors; K-EmoCon; multi modalities; transfer learning |
Similar Articles
| Id | Similarity | Authors | Title | Published |
|---|---|---|---|---|
5244  | 0.865 | Costa W.; Talavera E.; Oliveira R.; Figueiredo L.; Teixeira J.M.; Lima J.P.; Teichrieb V. | A Survey On Datasets For Emotion Recognition From Vision: Limitations And In-The-Wild Applicability | Applied Sciences (Switzerland), 13, 9 (2023) |
| 22600 | 0.859 | Nahin R.A.; Islam M.T.; Kabir A.; Afrin S.; Chowdhury I.A.; Rahman R.; Alam M.G.R. | Electroencephalogram-Based Emotion Recognition With Hybrid Graph Convolutional Network Model | 2023 IEEE 13th Annual Computing and Communication Workshop and Conference, CCWC 2023 (2023) |
| 23901 | 0.852 | Valen-Dacanay J.G. | Enhancing Proactive Immediate Response With Facial Emotion Detection Using Efficient Deep Learning | Lecture Notes in Networks and Systems, 1083 LNNS (2024) |