HS-AMM: Hierarchical Spatial Attention Mechanism Model for Facial Expression Recognition based on Deep Learning

Authors

• Ahmed Oday, Centre for Artificial Intelligence Technology, Faculty of Information Science and Technology, University Kebangsaan Malaysia, 43600 Bangi, Malaysia AND Department of Bioinformatics, College of Biomedical Informatics, University of Information Technology & Communications, 10001 Baghdad, IraqFollow
• Azizi Abdullah, Centre for Artificial Intelligence Technology, Faculty of Information Science and Technology, University Kebangsaan Malaysia, 43600 Bangi, MalaysiaFollow
• Shahnorbanun Sahran, Centre for Artificial Intelligence Technology, Faculty of Information Science and Technology, University Kebangsaan Malaysia, 43600 Bangi, MalaysiaFollow

Abstract

Facial expression recognition (FER) is a significant challenge due to the complexity of multi-class categorization and the subtle differences between emotional factors, with existing deep learning models often struggling to leverage hierarchical local features for accurate distinction. To find a solution, this study proposes a two-method approach, aiming to enhance FER accuracy by refining classification through the arousal-valence dimension and improving feature extraction via an adaptive attention mechanism to resolve the difficulty of differentiating semantically close expressions. The proposed methodology combines (1) a two-level classification strategy, first, performing fine-grained classification, and second, multi-class categorization by grouping expressions into neutral, positive, and negative classes based on emotional dimensions. (2) a hierarchical attention model that mitigates the challenge of fine-grained expression recognition by stacking global, regional, and local attention in a redesigned spatial hierarchy, first learning a robust semantic grouping (neutral, positive, and negative) of expressions. This approach reduces high-level confusion and provides a clearer pathway for subsequent fine-grained classification. This module extract emotion feature from facial images, all implemented on an enhanced VGG-16 backbone. The experiments on CK+, FER+, AffectNet, and RAF-basic datasets demonstrate that the proposed model achieves superior accuracy and robustness compared to existing methods, effectively capturing multi-level features and leveraging hierarchical subdivision for improved performance. The results confirm that our approach, through its fusion of multi-class grouping and hierarchical spatial attention, offers a more reliable and sophisticated solution for real-world emotion recognition applications.

Keywords

Facial expression recognition, Facial classification, Hierarchical attention, Multi-class, Spatial attention

Subject Area

Computer Science

Article Type

Article

First Page

1795

Last Page

1807

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite this Article

Oday, Ahmed; Abdullah, Azizi; and Sahran, Shahnorbanun (2026) "HS-AMM: Hierarchical Spatial Attention Mechanism Model for Facial Expression Recognition based on Deep Learning," Baghdad Science Journal: Vol. 23: Iss. 5, Article 17.
DOI: https://doi.org/10.21123/2411-7986.5301