A study of a vision-based lip movement analysis for hearing-impaired person
The study of vision based lip movement is the interpretation of human lip movement while speaking. Deaf and hard of hearing people often have a problem being able to understand what being talking in the conversation. Sign language may be useful for them to communicate but not everyone may underst...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Universiti Malaysia Perlis (UniMAP)
2019
|
| Subjects: | |
| Online Access: | http://dspace.unimap.edu.my:80/xmlui/handle/123456789/61991 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | The study of vision based lip movement is the interpretation of human lip movement
while speaking. Deaf and hard of hearing people often have a problem being able to
understand what being talking in the conversation. Sign language may be useful for
them to communicate but not everyone may understand the sign language. To make the
communication more interesting and does not have any obstacle to talk to anybody, the
lip reading is the best way to communicate. Deaf and hard hearing person can talk like
the normal person, however, they do not hear the word spoken by themselves. On the
other hand, there is no sound feedback system for them. This is a reason they could not
speak with the correct pronunciations. To teach them pronounce the word correctly,
fluently and to understand others in the conversation, the lip reading systems that could
train them pronounce the word correctly needs to be developed. Previous researcher
proposed many methods to recognize the spoken word based on the lip movement. One
of the method is attach the colour marker onto the lip surface and a vision sensor is used to track the moving marker. Other researchers extract the lip region by using Active
contour Model (ACM) or a snake method to draw the keypoint around lip edges. In this
studies, a system to track and recognize the spoken word based on lip movements is
proposed. A camera is used as a vision sensor and an image processing technique is
employed to extract and track the lip. The lip horizontal and vertical distances of the lip
are used to measure the ellipse while the systems track the lip movements. The
trajectories of the ellipse are resampled to 10 point called the features point. The word
database contains 10 spoken word frequently speak at the hospital are designed based
on the distribution of the 10 feature points. Two types of word database have been
designed which are the individual word database and the universal word database. The
individual word database is defined as the distributed feature points of each subject by
speaks the words with repetition. Meanwhile, the universal word database is defined as
the distributed feature point of all word regardless who is the subjects. The recognition
experiments are conducted and the system recognizes the unknown words with the
recognition rate 92.47% accuracy by using the individual database. Meanwhile, the
system recognizes the unknown words with the recognition rate 90.39% accuracy by
using the universal database. |
|---|