Cover Image

Comparative evaluation of anomaly-based controller area network IDS

The vulnerability of in-vehicle networks, particularly those based on the Controller Area Network (CAN) protocol, has prompted the development of numerous techniques for intrusion detection on the CAN bus. However, these CAN IDS are often evaluated in disparate experimental settings, with different...

Full description

Saved in:
Bibliographic Details
Main Authors: Sharmin, Shaila, Mansor, Hafizah, Abdul Kadir, Andi Fitriah, Abdul Aziz, Normaziah
Format: Conference or Workshop Item
Language:English
English
Published: Association for Computing Machinery 2023
Subjects:
QA75 Electronic computers. Computer science
Online Access:http://irep.iium.edu.my/105306/1/105306_Comparative%20evaluation%20of%20anomaly-based.pdf
http://irep.iium.edu.my/105306/7/105306_Comparative%20evaluation%20of%20anomaly-based_SCOPUS.pdf
http://irep.iium.edu.my/105306/
https://dl.acm.org/doi/10.1145/3587828.3587861
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.iium.irep.105306
record_format dspace
spelling my.iium.irep.1053062023-07-21T07:06:59Z http://irep.iium.edu.my/105306/ Comparative evaluation of anomaly-based controller area network IDS Sharmin, Shaila Mansor, Hafizah Abdul Kadir, Andi Fitriah Abdul Aziz, Normaziah QA75 Electronic computers. Computer science The vulnerability of in-vehicle networks, particularly those based on the Controller Area Network (CAN) protocol, has prompted the development of numerous techniques for intrusion detection on the CAN bus. However, these CAN IDS are often evaluated in disparate experimental settings, with different datasets and evaluation metrics, which hinder direct comparison. This has given rise to efforts at benchmarking and comparative evaluation of CAN IDS under similar experimental conditions to provide an understanding of the relative performance of these CAN IDS. This work contributes to these efforts by reporting results of the comparative evaluation of four statistical and two machine learning-based CAN intrusion detection algorithm, against the Real ORNL Automotive Dynamometer (ROAD) CAN intrusion dataset. The ROAD dataset differs from datasets used in previous work in that it includes the stealthiest possible version of targeted ID fabrication attacks which are more difficult to detect. It also consists of masquerade attacks, which have not been commonly used in comparative evaluation studies. Furthermore, in addition to metrics such as accuracy, precision, recall, and F1-score, we report balanced accuracy, informedness, markedness, and Matthews correlation coefficient, which place equal important on positive and negative classes and are better measures of detection capability, especially for imbalanced datasets. We also report training and testing times for each CAN IDS as an indicator of real-time intrusion detection performance. Results of experiments were found to be generally concordant with previous work, in terms of accuracy, precision, recall, and F1-score. Entropy and frequency-based CAN IDS were found to be relatively better at detecting attacks, particularly fabrication attacks; while other algorithms did not perform well, as indicated by low MCC scores. Association for Computing Machinery 2023-06-20 Conference or Workshop Item PeerReviewed application/pdf en http://irep.iium.edu.my/105306/1/105306_Comparative%20evaluation%20of%20anomaly-based.pdf application/pdf en http://irep.iium.edu.my/105306/7/105306_Comparative%20evaluation%20of%20anomaly-based_SCOPUS.pdf Sharmin, Shaila and Mansor, Hafizah and Abdul Kadir, Andi Fitriah and Abdul Aziz, Normaziah (2023) Comparative evaluation of anomaly-based controller area network IDS. In: ICSCA 2023: 2023 12th International Conference on Software and Computer Applications, 23rd - 25th February 2023, Kuantan, Malaysia. https://dl.acm.org/doi/10.1145/3587828.3587861 10.1145/3587828.3587861
institution Universiti Islam Antarabangsa Malaysia
building IIUM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider International Islamic University Malaysia
content_source IIUM Repository (IREP)
url_provider http://irep.iium.edu.my/
language English
English
topic QA75 Electronic computers. Computer science
spellingShingle QA75 Electronic computers. Computer science
Sharmin, Shaila
Mansor, Hafizah
Abdul Kadir, Andi Fitriah
Abdul Aziz, Normaziah
Comparative evaluation of anomaly-based controller area network IDS
description The vulnerability of in-vehicle networks, particularly those based on the Controller Area Network (CAN) protocol, has prompted the development of numerous techniques for intrusion detection on the CAN bus. However, these CAN IDS are often evaluated in disparate experimental settings, with different datasets and evaluation metrics, which hinder direct comparison. This has given rise to efforts at benchmarking and comparative evaluation of CAN IDS under similar experimental conditions to provide an understanding of the relative performance of these CAN IDS. This work contributes to these efforts by reporting results of the comparative evaluation of four statistical and two machine learning-based CAN intrusion detection algorithm, against the Real ORNL Automotive Dynamometer (ROAD) CAN intrusion dataset. The ROAD dataset differs from datasets used in previous work in that it includes the stealthiest possible version of targeted ID fabrication attacks which are more difficult to detect. It also consists of masquerade attacks, which have not been commonly used in comparative evaluation studies. Furthermore, in addition to metrics such as accuracy, precision, recall, and F1-score, we report balanced accuracy, informedness, markedness, and Matthews correlation coefficient, which place equal important on positive and negative classes and are better measures of detection capability, especially for imbalanced datasets. We also report training and testing times for each CAN IDS as an indicator of real-time intrusion detection performance. Results of experiments were found to be generally concordant with previous work, in terms of accuracy, precision, recall, and F1-score. Entropy and frequency-based CAN IDS were found to be relatively better at detecting attacks, particularly fabrication attacks; while other algorithms did not perform well, as indicated by low MCC scores.
format Conference or Workshop Item
author Sharmin, Shaila
Mansor, Hafizah
Abdul Kadir, Andi Fitriah
Abdul Aziz, Normaziah
author_facet Sharmin, Shaila
Mansor, Hafizah
Abdul Kadir, Andi Fitriah
Abdul Aziz, Normaziah
author_sort Sharmin, Shaila
title Comparative evaluation of anomaly-based controller area network IDS
title_short Comparative evaluation of anomaly-based controller area network IDS
title_full Comparative evaluation of anomaly-based controller area network IDS
title_fullStr Comparative evaluation of anomaly-based controller area network IDS
title_full_unstemmed Comparative evaluation of anomaly-based controller area network IDS
title_sort comparative evaluation of anomaly-based controller area network ids
publisher Association for Computing Machinery
publishDate 2023
url http://irep.iium.edu.my/105306/1/105306_Comparative%20evaluation%20of%20anomaly-based.pdf
http://irep.iium.edu.my/105306/7/105306_Comparative%20evaluation%20of%20anomaly-based_SCOPUS.pdf
http://irep.iium.edu.my/105306/
https://dl.acm.org/doi/10.1145/3587828.3587861
_version_ 1772810677047525376
score 13.211869

Similar Items