Cover Image

Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification

The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction,...

Full description

Saved in:
Bibliographic Details
Main Authors: Ashraf, Arselan, Gunawan, Teddy Surya, Arifin, Fatchul, Kartiwi, Mira, Sophian, Ali, Habaebi, Mohamed Hadi
Format: Article
Language:English
English
Published: Institute of Advanced Engineering and Science (IAES) 2023
Subjects:
TK7885 Computer engineering
Online Access:http://irep.iium.edu.my/105241/7/105241_Enhanced%20emotion%20recognition.pdf
http://irep.iium.edu.my/105241/8/105241_Enhanced%20emotion%20recognition_Scopus.pdf
http://irep.iium.edu.my/105241/
http://section.iaesonline.com/index.php/IJEEI/
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.iium.irep.105241
record_format dspace
spelling my.iium.irep.1052412023-06-27T03:56:32Z http://irep.iium.edu.my/105241/ Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification Ashraf, Arselan Gunawan, Teddy Surya Arifin, Fatchul Kartiwi, Mira Sophian, Ali Habaebi, Mohamed Hadi TK7885 Computer engineering The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction, machine learning, robotic control, and driver emotional state monitoring. With artificial intelligence and computational power advancements, visual emotion recognition has become a prominent research area. Despite extensive research employing machine learning algorithms like convolutional neural networks (CNN), challenges remain concerning input data processing, emotion classification scope, data size, optimal CNN configurations, and performance evaluation. To address these issues, we propose a comprehensive CNN-based model for real-time detection and classification of five primary emotions: anger, happiness, neutrality, sadness, and surprise. We employ the Amsterdam Dynamic Facial Expression Set – Bath Intensity Variations (ADFES-BIV) video dataset, extracting image frames from the video samples. Image processing techniques such as histogram equalization, color conversion, cropping, and resizing are applied to the frames before labeling. The Viola-Jones algorithm is then used for face detection on the processed grayscale images. We develop and train a CNN on the processed image data, implementing dropout, batch normalization, and L2 regularization to reduce overfitting. The ideal hyperparameters are determined through trial and error, and the model's performance is evaluated. The proposed model achieves a recognition accuracy of 99.38%, with the confusion matrix, recall, precision, F1 score, and processing time further quantifying its performance characteristics. The model's generalization performance is assessed using images from the Warsaw Set of Emotional Facial Expression Pictures (WSEFEP) and Extended Cohn-Kanade Database (CK+) datasets. The results demonstrate the efficiency and usability of our proposed approach, contributing valuable insights into real-time visual emotion recognition. Institute of Advanced Engineering and Science (IAES) 2023-03-24 Article PeerReviewed application/pdf en http://irep.iium.edu.my/105241/7/105241_Enhanced%20emotion%20recognition.pdf application/pdf en http://irep.iium.edu.my/105241/8/105241_Enhanced%20emotion%20recognition_Scopus.pdf Ashraf, Arselan and Gunawan, Teddy Surya and Arifin, Fatchul and Kartiwi, Mira and Sophian, Ali and Habaebi, Mohamed Hadi (2023) Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification. Indonesian Journal of Electrical Engineering and Informatics (IJEEI), 11 (1). pp. 286-299. ISSN 2089-3272 http://section.iaesonline.com/index.php/IJEEI/ 10.52549/ijeei.v11i1.4449
institution Universiti Islam Antarabangsa Malaysia
building IIUM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider International Islamic University Malaysia
content_source IIUM Repository (IREP)
url_provider http://irep.iium.edu.my/
language English
English
topic TK7885 Computer engineering
spellingShingle TK7885 Computer engineering
Ashraf, Arselan
Gunawan, Teddy Surya
Arifin, Fatchul
Kartiwi, Mira
Sophian, Ali
Habaebi, Mohamed Hadi
Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
description The human face is essential in conveying emotions, as facial expressions serve as effective, natural, and universal indicators of emotional states. Automated emotion recognition has garnered increasing interest due to its potential applications in various fields, such as human-computer interaction, machine learning, robotic control, and driver emotional state monitoring. With artificial intelligence and computational power advancements, visual emotion recognition has become a prominent research area. Despite extensive research employing machine learning algorithms like convolutional neural networks (CNN), challenges remain concerning input data processing, emotion classification scope, data size, optimal CNN configurations, and performance evaluation. To address these issues, we propose a comprehensive CNN-based model for real-time detection and classification of five primary emotions: anger, happiness, neutrality, sadness, and surprise. We employ the Amsterdam Dynamic Facial Expression Set – Bath Intensity Variations (ADFES-BIV) video dataset, extracting image frames from the video samples. Image processing techniques such as histogram equalization, color conversion, cropping, and resizing are applied to the frames before labeling. The Viola-Jones algorithm is then used for face detection on the processed grayscale images. We develop and train a CNN on the processed image data, implementing dropout, batch normalization, and L2 regularization to reduce overfitting. The ideal hyperparameters are determined through trial and error, and the model's performance is evaluated. The proposed model achieves a recognition accuracy of 99.38%, with the confusion matrix, recall, precision, F1 score, and processing time further quantifying its performance characteristics. The model's generalization performance is assessed using images from the Warsaw Set of Emotional Facial Expression Pictures (WSEFEP) and Extended Cohn-Kanade Database (CK+) datasets. The results demonstrate the efficiency and usability of our proposed approach, contributing valuable insights into real-time visual emotion recognition.
format Article
author Ashraf, Arselan
Gunawan, Teddy Surya
Arifin, Fatchul
Kartiwi, Mira
Sophian, Ali
Habaebi, Mohamed Hadi
author_facet Ashraf, Arselan
Gunawan, Teddy Surya
Arifin, Fatchul
Kartiwi, Mira
Sophian, Ali
Habaebi, Mohamed Hadi
author_sort Ashraf, Arselan
title Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
title_short Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
title_full Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
title_fullStr Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
title_full_unstemmed Enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
title_sort enhanced emotion recognition in videos: a convolutional neural network strategy for human facial expression detection and classification
publisher Institute of Advanced Engineering and Science (IAES)
publishDate 2023
url http://irep.iium.edu.my/105241/7/105241_Enhanced%20emotion%20recognition.pdf
http://irep.iium.edu.my/105241/8/105241_Enhanced%20emotion%20recognition_Scopus.pdf
http://irep.iium.edu.my/105241/
http://section.iaesonline.com/index.php/IJEEI/
_version_ 1769841838439530496
score 13.211869

Similar Items