Cover Image

Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence

Deeplearning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have...

Full description

Saved in:
Bibliographic Details
Main Authors: Ibrahim, MohdZamri, Saealal, Muhammad Salihin, J.Mulvaney, David., Shapiai, Mohd Ibrahim, Fadilah, Norasyikin
Format: Article
Language:English
Published: Public Library of Science CODEN 2022
Online Access:http://eprints.utem.edu.my/id/eprint/28189/2/0235514082024141210.pdf
http://eprints.utem.edu.my/id/eprint/28189/
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0278989
https://doi.org/10.1371/journal.pone.0278989
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.utem.eprints.28189
record_format eprints
spelling my.utem.eprints.281892025-01-06T10:55:22Z http://eprints.utem.edu.my/id/eprint/28189/ Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence Ibrahim, MohdZamri Saealal, Muhammad Salihin J.Mulvaney, David. Shapiai, Mohd Ibrahim Fadilah, Norasyikin Deeplearning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have been recently used to spread false news or disinformation. This study aims to identify Deepfaked videos and images and alert viewers to the possible falsity of the information. The current work presented a novel means of revealing fake face videos by cascading the convolution network with recurrent neural networks and fully connected network (FCN) models. The system detection approach utilizes the eye-blinking state in temporal video frames. Notwithstanding, it is deemed challenging to precisely depict (i) artificiality in fake videos and (ii) spatial information within the individual frame through this physiological signal. Spatial features were extracted using the VGG16 network and trained with the ImageNet dataset. The temporal features were then extracted in every 20 sequences through the LSTMnetwork. Onanother note, the pre-processed eye-blinking state served as a probability to generate a novel BPD dataset. This newly-acquired dataset wasfedtothree models for training purposes with each entailing four, three, and six hidden layers, respectively. Every model constitutes a unique architecture and specific dropout value. Resultantly, the model optimally and accurately identified tampered videos within the dataset. The study model was assessed using the current BPD dataset based on one of the most complex datasets (FaceForensic++) with 90.8% accuracy. Such precision was successfully maintained in datasets that were not used in the training process. The training process wasalso accelerated by lowering the computation prerequisites. Public Library of Science CODEN 2022-12 Article PeerReviewed text en cc_by_4 http://eprints.utem.edu.my/id/eprint/28189/2/0235514082024141210.pdf Ibrahim, MohdZamri and Saealal, Muhammad Salihin and J.Mulvaney, David. and Shapiai, Mohd Ibrahim and Fadilah, Norasyikin (2022) Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence. PLoS ONE, 17 (12). pp. 1-23. ISSN 1932-6203 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0278989 https://doi.org/10.1371/journal.pone.0278989
institution Universiti Teknikal Malaysia Melaka
building UTEM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Teknikal Malaysia Melaka
content_source UTEM Institutional Repository
url_provider http://eprints.utem.edu.my/
language English
description Deeplearning is notably successful in data analysis, computer vision, and human control. Nevertheless, this approach has inevitably allowed the development of DeepFake video sequences and images that could be altered so that the changes are not easily or explicitly detectable. Such alterations have been recently used to spread false news or disinformation. This study aims to identify Deepfaked videos and images and alert viewers to the possible falsity of the information. The current work presented a novel means of revealing fake face videos by cascading the convolution network with recurrent neural networks and fully connected network (FCN) models. The system detection approach utilizes the eye-blinking state in temporal video frames. Notwithstanding, it is deemed challenging to precisely depict (i) artificiality in fake videos and (ii) spatial information within the individual frame through this physiological signal. Spatial features were extracted using the VGG16 network and trained with the ImageNet dataset. The temporal features were then extracted in every 20 sequences through the LSTMnetwork. Onanother note, the pre-processed eye-blinking state served as a probability to generate a novel BPD dataset. This newly-acquired dataset wasfedtothree models for training purposes with each entailing four, three, and six hidden layers, respectively. Every model constitutes a unique architecture and specific dropout value. Resultantly, the model optimally and accurately identified tampered videos within the dataset. The study model was assessed using the current BPD dataset based on one of the most complex datasets (FaceForensic++) with 90.8% accuracy. Such precision was successfully maintained in datasets that were not used in the training process. The training process wasalso accelerated by lowering the computation prerequisites.
format Article
author Ibrahim, MohdZamri
Saealal, Muhammad Salihin
J.Mulvaney, David.
Shapiai, Mohd Ibrahim
Fadilah, Norasyikin
spellingShingle Ibrahim, MohdZamri
Saealal, Muhammad Salihin
J.Mulvaney, David.
Shapiai, Mohd Ibrahim
Fadilah, Norasyikin
Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
author_facet Ibrahim, MohdZamri
Saealal, Muhammad Salihin
J.Mulvaney, David.
Shapiai, Mohd Ibrahim
Fadilah, Norasyikin
author_sort Ibrahim, MohdZamri
title Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
title_short Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
title_full Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
title_fullStr Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
title_full_unstemmed Using cascade CNN-LSTM-FCNs toidentify AIaltered video based on eye state sequence
title_sort using cascade cnn-lstm-fcns toidentify aialtered video based on eye state sequence
publisher Public Library of Science CODEN
publishDate 2022
url http://eprints.utem.edu.my/id/eprint/28189/2/0235514082024141210.pdf
http://eprints.utem.edu.my/id/eprint/28189/
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0278989
https://doi.org/10.1371/journal.pone.0278989
_version_ 1821007592223145984
score 13.232414

Similar Items