Cover Image

Campus abnormal behavior recognition with temporal segment transformers

The intelligent campus surveillance system is beneficial to improve safety in school. Abnormal behavior recognition, a field of action recognition in computer vision, plays an essential role in intelligent surveillance systems. Computer vision has been actively applied to action recognition systems...

Full description

Saved in:
Bibliographic Details
Main Authors: Liu, Hai Chuan, Chuah, Joon Huang, Mohd Khairuddin, Anis Salwa, Zhao, Xian Min, Wang, Xiao Dan
Format: Article
Published: Institute of Electrical and Electronics Engineers 2023
Subjects:
TK Electrical engineering. Electronics Nuclear engineering
Online Access:http://eprints.um.edu.my/39040/
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.um.eprints.39040
record_format eprints
spelling my.um.eprints.390402023-07-04T06:32:34Z http://eprints.um.edu.my/39040/ Campus abnormal behavior recognition with temporal segment transformers Liu, Hai Chuan Chuah, Joon Huang Mohd Khairuddin, Anis Salwa Zhao, Xian Min Wang, Xiao Dan TK Electrical engineering. Electronics Nuclear engineering The intelligent campus surveillance system is beneficial to improve safety in school. Abnormal behavior recognition, a field of action recognition in computer vision, plays an essential role in intelligent surveillance systems. Computer vision has been actively applied to action recognition systems based on Convolutional Neural Networks (CNNs). However, capturing sufficient motion sequence features from videos remains a significant challenge in action recognition. This work explores the challenges of video-based abnormal behavior recognition on campus. In addition, a novel framework is established on long-range temporal video structure modeling and a global sparse uniform sampling strategy that divides a video into three segments of identical durations and uniformly samples each snippet. The proposed method incorporates a consensus of three temporal segment transformers (TST) that globally connects patches and computes self-attention with joint spatiotemporal factorization. The proposed model is developed on the newly created campus abnormal behavior recognition (CABR50) dataset, which contains 50 human abnormal action classes with an average of over 700 clips per class. Experiments show that it is feasible to implement abnormal behavior recognition on campus and that the proposed method is competitive with other peer video recognition in terms of Top-1 and Top-5 recognition accuracy. The results suggest that TST-L+ can improve campus abnormal behavior recognition, corresponding to Top-1 and Top-5 accuracy results of 83.57% and 97.16%, respectively. Institute of Electrical and Electronics Engineers 2023 Article PeerReviewed Liu, Hai Chuan and Chuah, Joon Huang and Mohd Khairuddin, Anis Salwa and Zhao, Xian Min and Wang, Xiao Dan (2023) Campus abnormal behavior recognition with temporal segment transformers. IEEE Access, 11. pp. 38471-38484. ISSN 2169-3536, DOI https://doi.org/10.1109/ACCESS.2023.3266440 <https://doi.org/10.1109/ACCESS.2023.3266440>. 10.1109/ACCESS.2023.3266440
institution Universiti Malaya
building UM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Malaya
content_source UM Research Repository
url_provider http://eprints.um.edu.my/
topic TK Electrical engineering. Electronics Nuclear engineering
spellingShingle TK Electrical engineering. Electronics Nuclear engineering
Liu, Hai Chuan
Chuah, Joon Huang
Mohd Khairuddin, Anis Salwa
Zhao, Xian Min
Wang, Xiao Dan
Campus abnormal behavior recognition with temporal segment transformers
description The intelligent campus surveillance system is beneficial to improve safety in school. Abnormal behavior recognition, a field of action recognition in computer vision, plays an essential role in intelligent surveillance systems. Computer vision has been actively applied to action recognition systems based on Convolutional Neural Networks (CNNs). However, capturing sufficient motion sequence features from videos remains a significant challenge in action recognition. This work explores the challenges of video-based abnormal behavior recognition on campus. In addition, a novel framework is established on long-range temporal video structure modeling and a global sparse uniform sampling strategy that divides a video into three segments of identical durations and uniformly samples each snippet. The proposed method incorporates a consensus of three temporal segment transformers (TST) that globally connects patches and computes self-attention with joint spatiotemporal factorization. The proposed model is developed on the newly created campus abnormal behavior recognition (CABR50) dataset, which contains 50 human abnormal action classes with an average of over 700 clips per class. Experiments show that it is feasible to implement abnormal behavior recognition on campus and that the proposed method is competitive with other peer video recognition in terms of Top-1 and Top-5 recognition accuracy. The results suggest that TST-L+ can improve campus abnormal behavior recognition, corresponding to Top-1 and Top-5 accuracy results of 83.57% and 97.16%, respectively.
format Article
author Liu, Hai Chuan
Chuah, Joon Huang
Mohd Khairuddin, Anis Salwa
Zhao, Xian Min
Wang, Xiao Dan
author_facet Liu, Hai Chuan
Chuah, Joon Huang
Mohd Khairuddin, Anis Salwa
Zhao, Xian Min
Wang, Xiao Dan
author_sort Liu, Hai Chuan
title Campus abnormal behavior recognition with temporal segment transformers
title_short Campus abnormal behavior recognition with temporal segment transformers
title_full Campus abnormal behavior recognition with temporal segment transformers
title_fullStr Campus abnormal behavior recognition with temporal segment transformers
title_full_unstemmed Campus abnormal behavior recognition with temporal segment transformers
title_sort campus abnormal behavior recognition with temporal segment transformers
publisher Institute of Electrical and Electronics Engineers
publishDate 2023
url http://eprints.um.edu.my/39040/
_version_ 1770551496773992448
score 13.211869

Similar Items