Efficient Multi-Class Facial Emotion Recognition using YOLOv9: A Deep Learning Approach for Real-Time Applications

doi:10.23940/ijpe.24.09.p6.581590

Abstract

Abstract: This work introduces a revolutionary YOLOv9 deep learning architecture-based method for facial emotion identification. Happy, Sad, Angry, Disgust, Natural, and Surprise are the six emotional classifications the research divides the 21,263 images into which the research divides. The model was trained on 88% of the dataset; 4% was used for testing and 8% for validation. The preprocessing operations were auto-orientation, zoom, rotation, and 416x416 pixel scaling. Training time was cut down significantly because the experiment was carried out with a T4 GPU from Kaggle. Five epochs later, the average mAP score of 0.85, average precision of 0.74, and average recall of 0.84 indicate encouraging emotional performance. At a mAP score of 0.98, the model showed exceptionally high accuracy in identifying disgust. With its reliable and effective approach to real-time emotion recognition in various applications, this work advances the field of affective computing.

Key words: computer vision, face emotion, deep learning, image processing

Ekta Singh, and Parma Nand. Efficient Multi-Class Facial Emotion Recognition using YOLOv9: A Deep Learning Approach for Real-Time Applications [J]. Int J Performability Eng, 2024, 20(9): 581-590.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

References

[1] Rawal, N. and Stock-Homburg, R.M., 2022. Facial emotion expressions in human-robot interaction: A survey. International Journal of Social Robotics,14(7), pp.1583-1604.
[2] Sharma, C. and Das, P., STUDY OF COMPUTER AIDED FACE RECOGNITION METHODS IN CONSTRAINED AND UNCONSTRAINED ENVIRONMENT.
[3] Naga P., Marri S.D. and Borreo R., 2023. Facial emotion recognition methods, datasets and technologies: A literature survey.Materials Today: Proceedings, 80, pp.2824-2828.
[4] Mendes C., Pereira R., Ribeiro J., Rodrigues N. and Pereira A., 2023, July. Chatto: An emotionally intelligent avatar for elderly care in ambient assisted living. In International Symposium on Ambient Intelligence(pp. 93-102). Cham: Springer Nature Switzerland.
[5] Van Thanh, N., 2023. Emotion recognition systems in retail: a detailed analysis of their role in enhancing customer interactions, driving sales, and predicting trends. Journal of Computational Social Dynamics,8(3), pp.1-9.
[6] Taye M.M.,2023. Understanding of machine learning with deep learning: architectures, workflow, applications and future directions.Computers, 12(5), p.91.
[7] Kamath, V. and Renuka, A., 2023. Deep learning based object detection for resource constrained devices: Systematic review, future trends and challenges ahead.Neurocomputing, 531, pp.34-60.
[8] Dujaili M.J.A.,2024. Survey on facial expressions recognition: databases, features and classification schemes. Multimedia Tools and Applications,83(3), pp.7457-7478.
[9] Liu S., Huang S., Fu W. and Lin J.C.W., 2024. A descriptive human visual cognitive strategy using graph neural network for facial expression recognition. International Journal of Machine Learning and Cybernetics,15(1), pp.19-35.
[10] Gong W., Qian Y., Zhou W. and Leng H., 2024. Enhanced spatial-temporal learning network for dynamic facial expression recognition. Biomedical Signal Processing and Control, 88, p.105316.
[11] Tao, H. and Duan, Q., 2024. Hierarchical attention network with progressive feature fusion for facial expression recognition.Neural Networks, 170, pp.337-348.
[12] Chen J., Shi J. and Xu R., 2024. Dual subspace manifold learning based on GCN for intensity-invariant facial expression recognition. Pattern Recognition, 148, p.110157.
[13] Brandt M., de Oliveira Silva F., Simões Neto J.P., Tourinho Baptista M.A., Belfort T., Lacerda I.B. and Nascimento Dourado M.C., 2024. Facial expression recognition of emotional situations in mild and moderate Alzheimer’s disease. Journal of Geriatric Psychiatry and Neurology,37(1), pp.73-83.
[14] Zhu A., Li K., Wu T., Zhao P., Zhou W. and Hong B., 2024. Cross-task multi-branch vision transformer for facial expression and mask wearing classification.arXiv preprint arXiv:2404.14606.
[15] Haq H.B.U., Akram W., Irshad M.N., Kosar A. and Abid M., 2024. Enhanced real-time facial expression recognition using deep learning. Acadlore Trans. Mach. Learn,3(1), pp.24-35.
[16] Face recognition. emotion detection Dataset. https://universe.roboflow.com/face-recognition-ixqtg/emotion-detection-cwq4g, accessed on May 15, 2024.
[17] Nair, V. and Kanojia, M., 2023. Integrating Facial Emotion Recognition into Music Recommendation Systems using YOLOv8. Journal of Information Assurance & Security,18(6).
[18] Ward T.,2023. Development of Detection and Tracking Systems for Autonomous Vehicles Using Machine Learning (Master's thesis, Morehead State University).
[19] Pérez-García F., Sparks R. and Ourselin S., 2021. TorchIO: a Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. Computer methods and programs in biomedicine, 208, p.106236.
[20] Alshahrani A., Almatrafi M.M., Mustafa J.I., Albaqami L.S. and Aljabri R.A., 2024. A Children's Psychological and Mental Health Detection Model by Drawing Analysis based on Computer Vision and Deep Learning. Engineering, Technology & Applied Science Research,14(4), pp.15533-15540.
[21] Pratama Y., Rasywir E., Sunoto A. and Irawan I., 2021. Application of yolo (you only look once) v. 4 with preprocessing image and network experiment. The IJICS (International Journal of Informatics and Computer Science),5(3), pp.280-286.
[22] Wang C.Y., Yeh I.H. and Liao H.Y.M., 2024. Yolov9: Learning what you want to learn using programmable gradient information.arXiv preprint arXiv:2402.13616.
[23] He T., Liu Y., Yu Y., Zhao Q. and Hu Z., 2020. Application of deep convolutional neural network on feature extraction and detection of wood defects. Measurement, 152, p.107357.
[24] Ultralytics YOLO Docs.YOLOv9: A Leap Forward in Object Detection Technology. https://docs.ultralytics.com/ models/yolov9/#performance-on-ms-coco-dataset, accessed on May 10, 2024.