• Sanskriti Rajiv Singh
• Aakash Anil Pasare
• Nikhil Gupta
• H.R. Kulkarni

Emotion recognition plays a significant role in human–computer interaction, mental health analysis, social robotics, and surveillance systems. Traditional emotion recognition methods primarily rely on single modalities such as facial expressions or speech. However, single-modality systems suffer from noise, occlusion, accent variations, and environmental disturbances. To overcome these challenges, this research proposes a Multimodal AI-based Emotion Recognition System that integrates facial expression analysis, speech features, and text sentiment to enhance accuracy and reliability. The model uses a combination of Convolutional Neural Networks (CNN) for visual features, Mel-Spectrogram + LSTM for audio analysis, and BERT for text-based sentiment extraction. The modalities are fused using a CMAF (Cross-Modal Attention Fusion) mechanism. The system is evaluated on multimodal datasets and shows improved accuracy compared to unimodal methods.

Emotion recognition plays a significant role in human–computer interaction, mental health analysis, social robotics, and surveillance systems. Traditional emotion recognition methods primarily rely on single modalities such as facial expressions or speech. However, single-modality systems suffer from noise, occlusion, accent variations, and environmental disturbances. To overcome these challenges, this research proposes a Multimodal AI-based Emotion Recognition System that integrates facial expression analysis, speech features, and text sentiment to enhance accuracy and reliability. The model uses a combination of Convolutional Neural Networks (CNN) for visual features, Mel-Spectrogram + LSTM for audio analysis, and BERT for text-based sentiment extraction. The modalities are fused using a CMAF (Cross-Modal Attention Fusion) mechanism. The system is evaluated on multimodal datasets and shows improved accuracy compared to unimodal methods.

"Emotion Recognition Using Multimodal AI", International Journal of Emerging Technologies and Innovative Research (www.jetir.org), ISSN:2349-5162, Vol.13, Issue 1, page no.20-27, January-2026, Available :http://www.jetir.org/papers/JETIRHG06002.pdf

"Emotion Recognition Using Multimodal AI", International Journal of Emerging Technologies and Innovative Research (www.jetir.org | UGC and issn Approved), ISSN:2349-5162, Vol.13, Issue 1, page no. pp20-27, January-2026, Available at : http://www.jetir.org/papers/JETIRHG06002.pdf