Cover Image

Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data

The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD...

Full description

Saved in:
Bibliographic Details
Main Authors: Siti Zahariah, Midi, Habshah
Format: Article
Published: Taylor and Francis 2022
Online Access:http://psasir.upm.edu.my/id/eprint/102186/
https://www.tandfonline.com/doi/abs/10.1080/02664763.2022.2093842?journalCode=cjas20
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.upm.eprints.102186
record_format eprints
spelling my.upm.eprints.1021862023-07-10T00:10:13Z http://psasir.upm.edu.my/id/eprint/102186/ Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data Siti Zahariah Midi, Habshah The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD-MRCD), which indicates a decrease in its performance as the number of independent variables (p) increases. The RMD-MRCD-PCA is developed by incorporating the Principal Component Analysis (PCA) in the MRCD algorithm whereby this robust approach shrinks the covariance matrix to make it invertible and thus, can be employed to compute the RMD for high dimensional data. A simulation study and two real data sets are used to illustrate the merit of our proposed method compared to the RMD-MRCD and Robust PCA (ROBPCA) methods. Findings show that the performance of the RMD-MRCD is similar to the performance of the RMD-MRCD-PCA for p close to 200. However, its performance tends to decrease when the number of p is more than 200 and worsens at p equals 700 and larger. On the other hand, the ROBPCA is not effective for less than 20% contamination as it suffers from serious swamping problems. Taylor and Francis 2022-07-08 Article PeerReviewed Siti Zahariah and Midi, Habshah (2022) Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data. Journal of Applied Statistics. pp. 1-19. ISSN 0266-4763 https://www.tandfonline.com/doi/abs/10.1080/02664763.2022.2093842?journalCode=cjas20 10.1080/02664763.2022.2093842
institution Universiti Putra Malaysia
building UPM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Putra Malaysia
content_source UPM Institutional Repository
url_provider http://psasir.upm.edu.my/
description The main aim of this paper is to propose a novel method (RMD-MRCD-PCA) of identification of High Leverage Points (HLPs) in high-dimensional sparse data. It is to address the weakness of the Robust Mahalanobis Distance (RMD) method which is based on the Minimum Regularized Covariance Determinant (RMD-MRCD), which indicates a decrease in its performance as the number of independent variables (p) increases. The RMD-MRCD-PCA is developed by incorporating the Principal Component Analysis (PCA) in the MRCD algorithm whereby this robust approach shrinks the covariance matrix to make it invertible and thus, can be employed to compute the RMD for high dimensional data. A simulation study and two real data sets are used to illustrate the merit of our proposed method compared to the RMD-MRCD and Robust PCA (ROBPCA) methods. Findings show that the performance of the RMD-MRCD is similar to the performance of the RMD-MRCD-PCA for p close to 200. However, its performance tends to decrease when the number of p is more than 200 and worsens at p equals 700 and larger. On the other hand, the ROBPCA is not effective for less than 20% contamination as it suffers from serious swamping problems.
format Article
author Siti Zahariah
Midi, Habshah
spellingShingle Siti Zahariah
Midi, Habshah
Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
author_facet Siti Zahariah
Midi, Habshah
author_sort Siti Zahariah
title Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
title_short Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
title_full Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
title_fullStr Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
title_full_unstemmed Minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
title_sort minimum regularized covariance determinant and principal component analysis-based method for the identification of high leverage points in high dimensional sparse data
publisher Taylor and Francis
publishDate 2022
url http://psasir.upm.edu.my/id/eprint/102186/
https://www.tandfonline.com/doi/abs/10.1080/02664763.2022.2093842?journalCode=cjas20
_version_ 1772813429275361280
score 13.244368

Similar Items