Cover Image

Hybrid fusion of face and speech information for bimodal emotion estimation

Estimation of human emotions during a conversation is difficult using a computer. In this study, facial expressions and speech are used in order to estimate emotions (angry, sad, happy, boredom, disgust and surprise). A proposed hybrid system through facial expressions and speech is used to estimate...

Full description

Saved in:
Bibliographic Details
Main Authors: Mohan Kudiri, K., Md. Said, A., Nayan, M.Y.
Format: Article
Published: Penerbit UTM Press 2016
Online Access:https://www.scopus.com/inward/record.uri?eid=2-s2.0-84988353604&doi=10.11113%2fjt.v78.9538&partnerID=40&md5=79e645383e5248b7358a33cf8bc1bc66
http://eprints.utp.edu.my/25487/
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.utp.eprints.25487
record_format eprints
spelling my.utp.eprints.254872021-08-27T13:02:07Z Hybrid fusion of face and speech information for bimodal emotion estimation Mohan Kudiri, K. Md. Said, A. Nayan, M.Y. Estimation of human emotions during a conversation is difficult using a computer. In this study, facial expressions and speech are used in order to estimate emotions (angry, sad, happy, boredom, disgust and surprise). A proposed hybrid system through facial expressions and speech is used to estimate emotions of a person when he is engaged in a conversational session. Relative Bin Frequency Coefficients and Relative Sub-Image-Based features are used for acoustic and visual modalities respectively. Support Vector Machine is used for classification. This study shows that the proposed feature extraction through acoustic and visual data is the most prominent aspect affecting the emotion detection system, along with the proposed fusion technique. Although some other aspects are considered to be affecting the system, the effect is relatively minor. It was observed that the performance of the bimodal system was lower than the unimodal system through deliberate facial expressions. In order to deal with the problem, a suitable database is used. The results indicate that the proposed system showed better performance, with respect to basic emotional classes than the rest. © 2016 Penerbit UTM Press. All rights reserved. Penerbit UTM Press 2016 Article NonPeerReviewed https://www.scopus.com/inward/record.uri?eid=2-s2.0-84988353604&doi=10.11113%2fjt.v78.9538&partnerID=40&md5=79e645383e5248b7358a33cf8bc1bc66 Mohan Kudiri, K. and Md. Said, A. and Nayan, M.Y. (2016) Hybrid fusion of face and speech information for bimodal emotion estimation. Jurnal Teknologi, 78 (8-2). pp. 26-34. http://eprints.utp.edu.my/25487/
institution Universiti Teknologi Petronas
building UTP Resource Centre
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Teknologi Petronas
content_source UTP Institutional Repository
url_provider http://eprints.utp.edu.my/
description Estimation of human emotions during a conversation is difficult using a computer. In this study, facial expressions and speech are used in order to estimate emotions (angry, sad, happy, boredom, disgust and surprise). A proposed hybrid system through facial expressions and speech is used to estimate emotions of a person when he is engaged in a conversational session. Relative Bin Frequency Coefficients and Relative Sub-Image-Based features are used for acoustic and visual modalities respectively. Support Vector Machine is used for classification. This study shows that the proposed feature extraction through acoustic and visual data is the most prominent aspect affecting the emotion detection system, along with the proposed fusion technique. Although some other aspects are considered to be affecting the system, the effect is relatively minor. It was observed that the performance of the bimodal system was lower than the unimodal system through deliberate facial expressions. In order to deal with the problem, a suitable database is used. The results indicate that the proposed system showed better performance, with respect to basic emotional classes than the rest. © 2016 Penerbit UTM Press. All rights reserved.
format Article
author Mohan Kudiri, K.
Md. Said, A.
Nayan, M.Y.
spellingShingle Mohan Kudiri, K.
Md. Said, A.
Nayan, M.Y.
Hybrid fusion of face and speech information for bimodal emotion estimation
author_facet Mohan Kudiri, K.
Md. Said, A.
Nayan, M.Y.
author_sort Mohan Kudiri, K.
title Hybrid fusion of face and speech information for bimodal emotion estimation
title_short Hybrid fusion of face and speech information for bimodal emotion estimation
title_full Hybrid fusion of face and speech information for bimodal emotion estimation
title_fullStr Hybrid fusion of face and speech information for bimodal emotion estimation
title_full_unstemmed Hybrid fusion of face and speech information for bimodal emotion estimation
title_sort hybrid fusion of face and speech information for bimodal emotion estimation
publisher Penerbit UTM Press
publishDate 2016
url https://www.scopus.com/inward/record.uri?eid=2-s2.0-84988353604&doi=10.11113%2fjt.v78.9538&partnerID=40&md5=79e645383e5248b7358a33cf8bc1bc66
http://eprints.utp.edu.my/25487/
_version_ 1738656737004617728
score 13.211869

Similar Items