Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems
With the development and advancement of ICST, data-driven technology such as the Internet of Things (IoT) and Smart Technology including Smart Energy Management Systems (SEMS) has become a trend in many regions and around the globe. There is no doubt that data quality and data quality problems are a...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | en |
| Published: |
ASPG
2024
|
| Subjects: | |
| Online Access: | http://eprints.uthm.edu.my/12053/1/J17656_80b6f91c92b238eb2b089aeba84ca04e.pdf http://eprints.uthm.edu.my/12053/ https://doi.org/10.54216/JISIoT.120204 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1833419690501734400 |
|---|---|
| author | Wong, Parh Yong A. M. Alduais, Nayef Omar, Nurul Aswa A. Mostafa, Salama H. Y. Saad, Abdul-Malik Nasser, Abdullah H. M. Ghanem, Waheed Ali |
| author_facet | Wong, Parh Yong A. M. Alduais, Nayef Omar, Nurul Aswa A. Mostafa, Salama H. Y. Saad, Abdul-Malik Nasser, Abdullah H. M. Ghanem, Waheed Ali |
| author_sort | Wong, Parh Yong |
| building | UTHM Library |
| collection | Institutional Repository |
| content_provider | Universiti Tun Hussein Onn Malaysia |
| content_source | UTHM Institutional Repository |
| continent | Asia |
| country | Malaysia |
| description | With the development and advancement of ICST, data-driven technology such as the Internet of Things (IoT) and Smart Technology including Smart Energy Management Systems (SEMS) has become a trend in many regions and around the globe. There is no doubt that data quality and data quality problems are among the most vital topics to be addressed for a successful application of IoT-based SEMS. Poor data in such major yet delicate systems will affect the quality of life (QoL) of millions, and even cause destruction and disruption to a
country. This paper aims to tackle this problem by searching for suitable outlier detection techniques from the many developed ML-based outlier detection methods. Three methods are chosen and analyzed for their performances, namely the K-Nearest Neighbour (KNN)+ Mahalanobis Distance (MD), Minimum Covariance Determinant (MCD), and Local Outlier Factor (LOF) models. Three sensor-collected datasets that are related to SEMS and with different data types are used in this research, they are pre-processed and split into training and testing datasets with manually injected outliers. The training datasets are then used for searching the
patterns of the datasets through training of the models, and the trained models are then tested with the testing datasets, using the found patterns to identify and label the outliers in the datasets. All the models can accurately identify the outliers, with their average accuracies scoring over 95%. However, the average execution time used for each model varies, where the KNN+MD model has the longest average execution time at 12.99 seconds, MCD achieving 3.98 seconds for execution time, and the LOF model at 0.60 seconds, the
shortest among the three. |
| format | Article |
| id | my.uthm.eprints-12053 |
| institution | Universiti Tun Hussein Onn Malaysia |
| language | en |
| publishDate | 2024 |
| publisher | ASPG |
| record_format | eprints |
| spelling | my.uthm.eprints-120532025-05-02T07:07:07Z http://eprints.uthm.edu.my/12053/ Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems Wong, Parh Yong A. M. Alduais, Nayef Omar, Nurul Aswa A. Mostafa, Salama H. Y. Saad, Abdul-Malik Nasser, Abdullah H. M. Ghanem, Waheed Ali QA Mathematics With the development and advancement of ICST, data-driven technology such as the Internet of Things (IoT) and Smart Technology including Smart Energy Management Systems (SEMS) has become a trend in many regions and around the globe. There is no doubt that data quality and data quality problems are among the most vital topics to be addressed for a successful application of IoT-based SEMS. Poor data in such major yet delicate systems will affect the quality of life (QoL) of millions, and even cause destruction and disruption to a country. This paper aims to tackle this problem by searching for suitable outlier detection techniques from the many developed ML-based outlier detection methods. Three methods are chosen and analyzed for their performances, namely the K-Nearest Neighbour (KNN)+ Mahalanobis Distance (MD), Minimum Covariance Determinant (MCD), and Local Outlier Factor (LOF) models. Three sensor-collected datasets that are related to SEMS and with different data types are used in this research, they are pre-processed and split into training and testing datasets with manually injected outliers. The training datasets are then used for searching the patterns of the datasets through training of the models, and the trained models are then tested with the testing datasets, using the found patterns to identify and label the outliers in the datasets. All the models can accurately identify the outliers, with their average accuracies scoring over 95%. However, the average execution time used for each model varies, where the KNN+MD model has the longest average execution time at 12.99 seconds, MCD achieving 3.98 seconds for execution time, and the LOF model at 0.60 seconds, the shortest among the three. ASPG 2024 Article PeerReviewed text en http://eprints.uthm.edu.my/12053/1/J17656_80b6f91c92b238eb2b089aeba84ca04e.pdf Wong, Parh Yong and A. M. Alduais, Nayef and Omar, Nurul Aswa and A. Mostafa, Salama and H. Y. Saad, Abdul-Malik and Nasser, Abdullah and H. M. Ghanem, Waheed Ali (2024) Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems. Journal of Intelligent Systems and Internet of Things, 12 (2). pp. 44-64. https://doi.org/10.54216/JISIoT.120204 |
| spellingShingle | QA Mathematics Wong, Parh Yong A. M. Alduais, Nayef Omar, Nurul Aswa A. Mostafa, Salama H. Y. Saad, Abdul-Malik Nasser, Abdullah H. M. Ghanem, Waheed Ali Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title | Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title_full | Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title_fullStr | Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title_full_unstemmed | Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title_short | Comparative Analysis of ML-Based Outlier Detection Techniques for IoT-Based Smart Energy Management Systems |
| title_sort | comparative analysis of ml-based outlier detection techniques for iot-based smart energy management systems |
| topic | QA Mathematics |
| url | http://eprints.uthm.edu.my/12053/1/J17656_80b6f91c92b238eb2b089aeba84ca04e.pdf http://eprints.uthm.edu.my/12053/ https://doi.org/10.54216/JISIoT.120204 |
| url_provider | http://eprints.uthm.edu.my/ |