Cover Image

Ensemble-based face expression recognition approach for image sentiment analysis

Sentiment analysis based on images is an evolving area of study. Developing a reliable facial expression recognition (FER) device remains a difficult challenge as recognizing emotional feelings reflected in an image is dependent on a diverse set of factors. This paper presented an ensemble-based mod...

Full description

Saved in:
Bibliographic Details
Main Authors: Ervin Gubin Moung, Chai, Chuan Wooi, Maisarah Mohd Sufian, Chin Kim On
Format: Article
Language:English
English
Published: Yogyakarta: Institute of Advanced Engineering and Science (IAES) 2022
Subjects:
BF1-990 Psychology
QA71-90 Instruments and machines
Online Access:https://eprints.ums.edu.my/id/eprint/33628/1/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.pdf
https://eprints.ums.edu.my/id/eprint/33628/2/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.ABSTRACT.pdf
https://eprints.ums.edu.my/id/eprint/33628/
https://ijece.iaescore.com/index.php/IJECE/article/view/26411/15635%20https:/www.scopus.com/record/display.uri?eid=2-s2.0-85124998658&origin=resultslist&sort=plf-f
https://doi.org/10.11591/ijece.v12i3.pp2588-2600
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.ums.eprints.33628
record_format eprints
spelling my.ums.eprints.336282022-08-02T04:04:02Z https://eprints.ums.edu.my/id/eprint/33628/ Ensemble-based face expression recognition approach for image sentiment analysis Ervin Gubin Moung Chai, Chuan Wooi Maisarah Mohd Sufian Chin Kim On BF1-990 Psychology QA71-90 Instruments and machines Sentiment analysis based on images is an evolving area of study. Developing a reliable facial expression recognition (FER) device remains a difficult challenge as recognizing emotional feelings reflected in an image is dependent on a diverse set of factors. This paper presented an ensemble-based model for FER that incorporates multiple classification models: i) customized convolutional neural network (CNN), ii) ResNet50, and iii) InceptionV3. The model averaging ensemble classifier method is used to ensemble the predictions from the three models. Subsequently, the proposed FER model is trained and tested on a dataset with an uncontrolled environment (FER-2013 dataset). The experiment demonstrated that assembling multiple classifiers outperformed all single classifiers in classifying positive and neutral expressions (91.7%, 81.7% and 76.5% accuracy rate for happy, surprise, and neutral, respectively). However, when classifying disgust, anger, and sadness, the ResNet50 model alone is the better choice. Although the Custom CNN performs the best in classifying fear expression (55.7% accuracy), the proposed FER model can still classify fear expression with comparable performance (52.8% accuracy). This paper demonstrated the potential of using the ensemble-based method to enhance the performance of FER. As a result, the proposed FER model has shown a 72.3% accuracy rate. Yogyakarta: Institute of Advanced Engineering and Science (IAES) 2022-06 Article PeerReviewed text en https://eprints.ums.edu.my/id/eprint/33628/1/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.pdf text en https://eprints.ums.edu.my/id/eprint/33628/2/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.ABSTRACT.pdf Ervin Gubin Moung and Chai, Chuan Wooi and Maisarah Mohd Sufian and Chin Kim On (2022) Ensemble-based face expression recognition approach for image sentiment analysis. International Journal of Electrical and Computer Engineering (IJECE), 12 (3). pp. 2588-2600. ISSN 088-8708 https://ijece.iaescore.com/index.php/IJECE/article/view/26411/15635%20https:/www.scopus.com/record/display.uri?eid=2-s2.0-85124998658&origin=resultslist&sort=plf-f https://doi.org/10.11591/ijece.v12i3.pp2588-2600
institution Universiti Malaysia Sabah
building UMS Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Malaysia Sabah
content_source UMS Institutional Repository
url_provider http://eprints.ums.edu.my/
language English
English
topic BF1-990 Psychology
QA71-90 Instruments and machines
spellingShingle BF1-990 Psychology
QA71-90 Instruments and machines
Ervin Gubin Moung
Chai, Chuan Wooi
Maisarah Mohd Sufian
Chin Kim On
Ensemble-based face expression recognition approach for image sentiment analysis
description Sentiment analysis based on images is an evolving area of study. Developing a reliable facial expression recognition (FER) device remains a difficult challenge as recognizing emotional feelings reflected in an image is dependent on a diverse set of factors. This paper presented an ensemble-based model for FER that incorporates multiple classification models: i) customized convolutional neural network (CNN), ii) ResNet50, and iii) InceptionV3. The model averaging ensemble classifier method is used to ensemble the predictions from the three models. Subsequently, the proposed FER model is trained and tested on a dataset with an uncontrolled environment (FER-2013 dataset). The experiment demonstrated that assembling multiple classifiers outperformed all single classifiers in classifying positive and neutral expressions (91.7%, 81.7% and 76.5% accuracy rate for happy, surprise, and neutral, respectively). However, when classifying disgust, anger, and sadness, the ResNet50 model alone is the better choice. Although the Custom CNN performs the best in classifying fear expression (55.7% accuracy), the proposed FER model can still classify fear expression with comparable performance (52.8% accuracy). This paper demonstrated the potential of using the ensemble-based method to enhance the performance of FER. As a result, the proposed FER model has shown a 72.3% accuracy rate.
format Article
author Ervin Gubin Moung
Chai, Chuan Wooi
Maisarah Mohd Sufian
Chin Kim On
author_facet Ervin Gubin Moung
Chai, Chuan Wooi
Maisarah Mohd Sufian
Chin Kim On
author_sort Ervin Gubin Moung
title Ensemble-based face expression recognition approach for image sentiment analysis
title_short Ensemble-based face expression recognition approach for image sentiment analysis
title_full Ensemble-based face expression recognition approach for image sentiment analysis
title_fullStr Ensemble-based face expression recognition approach for image sentiment analysis
title_full_unstemmed Ensemble-based face expression recognition approach for image sentiment analysis
title_sort ensemble-based face expression recognition approach for image sentiment analysis
publisher Yogyakarta: Institute of Advanced Engineering and Science (IAES)
publishDate 2022
url https://eprints.ums.edu.my/id/eprint/33628/1/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.pdf
https://eprints.ums.edu.my/id/eprint/33628/2/Ensemble-based%20face%20expression%20recognition%20approach%20for%20image%20sentiment%20analysis.ABSTRACT.pdf
https://eprints.ums.edu.my/id/eprint/33628/
https://ijece.iaescore.com/index.php/IJECE/article/view/26411/15635%20https:/www.scopus.com/record/display.uri?eid=2-s2.0-85124998658&origin=resultslist&sort=plf-f
https://doi.org/10.11591/ijece.v12i3.pp2588-2600
_version_ 1760231188348272640
score 13.211869

Similar Items