Cover Image

NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT

Service ro~ots; .re preVl'I.Iling in many industries to assis~ humans in c~nduc,ing repetitive task.;, which require a natural interaction called Human Robot Interaction (HRI). In particular, nonverbal HRI plays an important role in social interactions, which highlights the need to accurately d...

Full description

Saved in:
Bibliographic Details
Main Authors: SOOMRO, ZUBAIR ADIL, SHAMSUDIN, ABU UBAIDAH, ABDUl RAHIM, RUZAIRI, ADRIAN, ANDI, HAZELI, MOHD
Format: Article
Language:English
Published: 2023
Subjects:
T Technology (General)
Online Access:http://eprints.uthm.edu.my/8300/1/J15662_942c3ecbc3be675cdaa9744d7645b4b4.pdf
http://eprints.uthm.edu.my/8300/
https://doi.org/10.31436/iiumej v:4i 1.2577
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.uthm.eprints.8300
record_format eprints
spelling my.uthm.eprints.83002023-02-14T08:22:01Z http://eprints.uthm.edu.my/8300/ NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT SOOMRO, ZUBAIR ADIL SHAMSUDIN, ABU UBAIDAH ABDUl RAHIM, RUZAIRI ADRIAN, ANDI HAZELI, MOHD T Technology (General) Service ro~ots; .re preVl'I.Iling in many industries to assis~ humans in c~nduc,ing repetitive task.;, which require a natural interaction called Human Robot Interaction (HRI). In particular, nonverbal HRI plays an important role in social interactions, which highlights the need to accurately detect the subject's attention by evaluating the programmed cues. 1n this paper, a conceptual attentiveness model algorithm called Attentive Recognition Model (ARM) is proposed to recognize a person's aLi:~ntivencss, which improves tne a-::Jtac~· of detection and subjective experience during nonverbal dRI using three combined detection models: face tracking, iris tracking and eye blin:.:ing. The face tracking model was trained using a Long Short-Term Memory (LSTM) neural network, which is based on deep learning. Meanwhile, the iris tracking and eye blinking use a mathematical model. The eye blinking model uses a random face landmark point to calculate the Eye Aspect Ratio (EAR), which is much more reliable compared to the prior method, which could detect a person blinking at a further distance even if the person was not blinking. The conducted e~ periments for face and iris tracking were able to detect direction up to 2 meters. Meanwhile, the tested eye blinking model gave an accuracy of 83.33% at up to 2 meters. The overall attentive accuracy of ARM was up to 85.7%. The experin1ents showed that the service robot was able to understand the programmed cues and hence perform certain tasks, such as approaching the interested person. Robot perkhidmatan lazim dalam banyak industri untuk membantu manusia menjalankan tugas berulang, yang memerlukan interaksi semula jadi yang dipanggil Interaksi Robot Manusia (HRI). Khususnya, HRI bukan lisan memainkan peranan penting dalam interaksi sosial, yang menonjolkan keperluan untuk mengesan perhatian subjek dengan tepat dengan menilai isyarat yang diprogramkan. Dalam makalah ini, algoritma model perhatian konseptual yang dipanggil Model Pengecaman Perhatian (ARM) dicadangkan untuk mengenali perhatian seseorang, yang meningkatkan ketepatan pengesanan dan pengalaman subjektif semasa HRI bukan lisan menggunakan tiga model pengesanan gabungan: pengesanan muka, pengesanan iris dan mata berkedip .. Model penjejakan muka telah dilatih menggunakan rangkaian saraf Memori Jangka Pendek Panjang (LSTM), yang berdasarkan pembelajaran mendalam. Manakala, pengesanan iris dan mata berkelip menggunakan model matematik. Model mata berkelip menggunakan titik mercu tanda muka rawak untuk mengira Nisbah Aspek Mata (EAR), yangjauh lebih dipercayai berbanding kaedah sebelumnya, yang boleh mengesan seseorang berkelip pada jarak yang lebih jauh walaupun orang itu tidak berkelip. Eksperin1en yang dijalankan untuk pengesanan muka dan iris dapat mengesan arah sehingga 2 meter. Sementara itu, model berkelip mata yang diuji memberikan ketepatan 2023 Article PeerReviewed text en http://eprints.uthm.edu.my/8300/1/J15662_942c3ecbc3be675cdaa9744d7645b4b4.pdf SOOMRO, ZUBAIR ADIL and SHAMSUDIN, ABU UBAIDAH and ABDUl RAHIM, RUZAIRI and ADRIAN, ANDI and HAZELI, MOHD (2023) NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT. IIUM Engineering Journal, 24 (1). pp. 301-318. https://doi.org/10.31436/iiumej v:4i 1.2577
institution Universiti Tun Hussein Onn Malaysia
building UTHM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Tun Hussein Onn Malaysia
content_source UTHM Institutional Repository
url_provider http://eprints.uthm.edu.my/
language English
topic T Technology (General)
spellingShingle T Technology (General)
SOOMRO, ZUBAIR ADIL
SHAMSUDIN, ABU UBAIDAH
ABDUl RAHIM, RUZAIRI
ADRIAN, ANDI
HAZELI, MOHD
NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
description Service ro~ots; .re preVl'I.Iling in many industries to assis~ humans in c~nduc,ing repetitive task.;, which require a natural interaction called Human Robot Interaction (HRI). In particular, nonverbal HRI plays an important role in social interactions, which highlights the need to accurately detect the subject's attention by evaluating the programmed cues. 1n this paper, a conceptual attentiveness model algorithm called Attentive Recognition Model (ARM) is proposed to recognize a person's aLi:~ntivencss, which improves tne a-::Jtac~· of detection and subjective experience during nonverbal dRI using three combined detection models: face tracking, iris tracking and eye blin:.:ing. The face tracking model was trained using a Long Short-Term Memory (LSTM) neural network, which is based on deep learning. Meanwhile, the iris tracking and eye blinking use a mathematical model. The eye blinking model uses a random face landmark point to calculate the Eye Aspect Ratio (EAR), which is much more reliable compared to the prior method, which could detect a person blinking at a further distance even if the person was not blinking. The conducted e~ periments for face and iris tracking were able to detect direction up to 2 meters. Meanwhile, the tested eye blinking model gave an accuracy of 83.33% at up to 2 meters. The overall attentive accuracy of ARM was up to 85.7%. The experin1ents showed that the service robot was able to understand the programmed cues and hence perform certain tasks, such as approaching the interested person. Robot perkhidmatan lazim dalam banyak industri untuk membantu manusia menjalankan tugas berulang, yang memerlukan interaksi semula jadi yang dipanggil Interaksi Robot Manusia (HRI). Khususnya, HRI bukan lisan memainkan peranan penting dalam interaksi sosial, yang menonjolkan keperluan untuk mengesan perhatian subjek dengan tepat dengan menilai isyarat yang diprogramkan. Dalam makalah ini, algoritma model perhatian konseptual yang dipanggil Model Pengecaman Perhatian (ARM) dicadangkan untuk mengenali perhatian seseorang, yang meningkatkan ketepatan pengesanan dan pengalaman subjektif semasa HRI bukan lisan menggunakan tiga model pengesanan gabungan: pengesanan muka, pengesanan iris dan mata berkedip .. Model penjejakan muka telah dilatih menggunakan rangkaian saraf Memori Jangka Pendek Panjang (LSTM), yang berdasarkan pembelajaran mendalam. Manakala, pengesanan iris dan mata berkelip menggunakan model matematik. Model mata berkelip menggunakan titik mercu tanda muka rawak untuk mengira Nisbah Aspek Mata (EAR), yangjauh lebih dipercayai berbanding kaedah sebelumnya, yang boleh mengesan seseorang berkelip pada jarak yang lebih jauh walaupun orang itu tidak berkelip. Eksperin1en yang dijalankan untuk pengesanan muka dan iris dapat mengesan arah sehingga 2 meter. Sementara itu, model berkelip mata yang diuji memberikan ketepatan
format Article
author SOOMRO, ZUBAIR ADIL
SHAMSUDIN, ABU UBAIDAH
ABDUl RAHIM, RUZAIRI
ADRIAN, ANDI
HAZELI, MOHD
author_facet SOOMRO, ZUBAIR ADIL
SHAMSUDIN, ABU UBAIDAH
ABDUl RAHIM, RUZAIRI
ADRIAN, ANDI
HAZELI, MOHD
author_sort SOOMRO, ZUBAIR ADIL
title NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
title_short NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
title_full NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
title_fullStr NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
title_full_unstemmed NON-VERBAL HUMAN-ROBOT INTERACTION USING NEURAL NETWORK FOR THE APPLICATION OF SERVICE ROBOT
title_sort non-verbal human-robot interaction using neural network for the application of service robot
publishDate 2023
url http://eprints.uthm.edu.my/8300/1/J15662_942c3ecbc3be675cdaa9744d7645b4b4.pdf
http://eprints.uthm.edu.my/8300/
https://doi.org/10.31436/iiumej v:4i 1.2577
_version_ 1758580268881412096
score 13.211869

Similar Items