Live facial expression recognition - Tan, Wei Mun (2022)

(untitled)
When having a conversation with our friends, facial expression plays an essential role in conveying emotions, unrevealed messages, and unconscious thoughts. By reading others' facial expressions in conservation, we could better understand how they feel towards the incidents and relate ourselves to others' speech or discover hidden information that the author does not want to reveal to us. However, people with barriers like autistic symptoms or visual impairments could have a hard time understanding others' facial expressions, which could provide them with additional information while communicating with others. Hence, this project is motivated by the will to help people with disabilities recognize others' facial expressions, helping them better communicate with others. The first draft design integrates an online FER API into a mobile application to help the target audience recognize others' emotions. However, several issues identified in the research are the high number of queries to a paid FER service, a high amount of data transmission when sending all the camera input frames to the online FER service and the relatively long overall processing time to recognize facial expressions from the original images captured by the smartphone. Therefore, this paper proposed three stages of potential optimizations: down sampling to reduce the number of camera frames to be processed, spatial trimming on the image to be sent to the FER service, and caching methodologies to reduce the number of FER queries that cost. Finally, the potential optimizations will be evaluated and determine the best implementation to produce a FER procedure that sends the appropriate frames only to be recognized and reduce the amount of data transmission during the FER query and perhaps improve the overall processing time from acquiring the input to delivering the FER result. The determined implementation will be realized by building the mobile application, improving the portability in FER.