Advanced CNN architectures and explainable AI for complex emotion recognition from facial images: experimental validation in human-robot interaction

Abstract

This study investigates the recognition of complex emotional states from facial images using advanced convolutional neural network
architectures and explainable artificial intelligence techniques. Unlike prior work focused on basic categorical emotions, we target subtle affective states such as frustration, confusion, or skepticism, which are critical for nuanced human-robot interactions. We compare a conventional deep learning model (ResNet50), an advanced EfficientNet-Transformer architecture, and our proposed CNN model enhanced with the Attention Map Alignment Layer (AMAL), designed to improve interpretability and focus on semantically relevant facial regions. Experimental evaluation on benchmark datasets (AffectNet, EMOTIC) and in a real-time simulation involving the OhBot social robot demonstrates that the proposed model achieves higher recognition accuracy for complex emotions and provides more consistent feature attribution using SHAP and LIME frameworks.
The results highlight the potential of integrating explainable computer vision systems into interactive robotics, improving transparency and emotional understanding in artificial agents.