Cover Image

Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence

Deep learning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have...

Full description

Saved in:
Bibliographic Details
Main Authors: Muhammad Salihin, Saealal, Mohd Zamri, Ibrahim, Mulvaney, D. J., Mohd Ibrahim, Shapiai, Norasyikin, Fadilah
Format: Article
Language:English
Published: Public Library of Science 2022
Subjects:
QA75 Electronic computers. Computer science
Online Access:http://umpir.ump.edu.my/id/eprint/39129/1/journal.pone.0278989%20%282%29.pdf
http://umpir.ump.edu.my/id/eprint/39129/
https://doi.org/10.1371/journal.pone.0278989
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.ump.umpir.39129
record_format eprints
spelling my.ump.umpir.391292023-11-01T07:46:33Z http://umpir.ump.edu.my/id/eprint/39129/ Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence Muhammad Salihin, Saealal Mohd Zamri, Ibrahim Mulvaney, D. J. Mohd Ibrahim, Shapiai Norasyikin, Fadilah QA75 Electronic computers. Computer science Deep learning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have been recently used to spread false news or disinformation. This study aims to identify Deepfaked videos and images and alert viewers to the possible falsity of the information. The current work presented a novel means of revealing fake face videos by cascading the convolution network with recurrent neural networks and fully connected network (FCN) models. The system detection approach utilizes the eye-blinking state in temporal video frames. Notwithstanding, it is deemed challenging to precisely depict (i) artificiality in fake videos and (ii) spatial information within the individual frame through this physiological signal. Spatial features were extracted using the VGG16 network and trained with the ImageNet dataset. The temporal features were then extracted in every 20 sequences through the LSTM network. On another note, the pre-processed eye-blinking state served as a probability to generate a novel BPD dataset. This newly-acquired dataset was fed to three models for training purposes with each entailing four, three, and six hidden layers, respectively. Every model constitutes a unique architecture and specific dropout value. Resultantly, the model optimally and accurately identified tampered videos within the dataset. The study model was assessed using the current BPD dataset based on one of the most complex datasets (FaceForensic++) with 90.8% accuracy. Such precision was successfully maintained in datasets that were not used in the training process. The training process was also accelerated by lowering the computation prerequisites. Public Library of Science 2022 Article PeerReviewed pdf en cc_by_4 http://umpir.ump.edu.my/id/eprint/39129/1/journal.pone.0278989%20%282%29.pdf Muhammad Salihin, Saealal and Mohd Zamri, Ibrahim and Mulvaney, D. J. and Mohd Ibrahim, Shapiai and Norasyikin, Fadilah (2022) Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence. PLoS ONE, 17 (12). ISSN 1932-6203. (Published) https://doi.org/10.1371/journal.pone.0278989 10.1371/journal.pone.0278989
institution Universiti Malaysia Pahang Al-Sultan Abdullah
building UMPSA Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Malaysia Pahang Al-Sultan Abdullah
content_source UMPSA Institutional Repository
url_provider http://umpir.ump.edu.my/
language English
topic QA75 Electronic computers. Computer science
spellingShingle QA75 Electronic computers. Computer science
Muhammad Salihin, Saealal
Mohd Zamri, Ibrahim
Mulvaney, D. J.
Mohd Ibrahim, Shapiai
Norasyikin, Fadilah
Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
description Deep learning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have been recently used to spread false news or disinformation. This study aims to identify Deepfaked videos and images and alert viewers to the possible falsity of the information. The current work presented a novel means of revealing fake face videos by cascading the convolution network with recurrent neural networks and fully connected network (FCN) models. The system detection approach utilizes the eye-blinking state in temporal video frames. Notwithstanding, it is deemed challenging to precisely depict (i) artificiality in fake videos and (ii) spatial information within the individual frame through this physiological signal. Spatial features were extracted using the VGG16 network and trained with the ImageNet dataset. The temporal features were then extracted in every 20 sequences through the LSTM network. On another note, the pre-processed eye-blinking state served as a probability to generate a novel BPD dataset. This newly-acquired dataset was fed to three models for training purposes with each entailing four, three, and six hidden layers, respectively. Every model constitutes a unique architecture and specific dropout value. Resultantly, the model optimally and accurately identified tampered videos within the dataset. The study model was assessed using the current BPD dataset based on one of the most complex datasets (FaceForensic++) with 90.8% accuracy. Such precision was successfully maintained in datasets that were not used in the training process. The training process was also accelerated by lowering the computation prerequisites.
format Article
author Muhammad Salihin, Saealal
Mohd Zamri, Ibrahim
Mulvaney, D. J.
Mohd Ibrahim, Shapiai
Norasyikin, Fadilah
author_facet Muhammad Salihin, Saealal
Mohd Zamri, Ibrahim
Mulvaney, D. J.
Mohd Ibrahim, Shapiai
Norasyikin, Fadilah
author_sort Muhammad Salihin, Saealal
title Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
title_short Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
title_full Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
title_fullStr Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
title_full_unstemmed Using cascade CNN-LSTM-FCNs to identify AI-altered video based on eye state sequence
title_sort using cascade cnn-lstm-fcns to identify ai-altered video based on eye state sequence
publisher Public Library of Science
publishDate 2022
url http://umpir.ump.edu.my/id/eprint/39129/1/journal.pone.0278989%20%282%29.pdf
http://umpir.ump.edu.my/id/eprint/39129/
https://doi.org/10.1371/journal.pone.0278989
_version_ 1822923829636235264
score 13.232414

Similar Items