Cover Image

Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review

Many recent studies have shown that various multi-objective evolutionary algorithms have been widely applied in the field of search-based software engineering (SBSE) for optimal solutions. Most of them either focused on solving newly re-formulated problems or on proposing new approaches, while a num...

Full description

Saved in:
Bibliographic Details
Main Authors: Nuh, Jamal Abdullahi, Koh, Tieng Wei, Baharom, Salmi, Osman, Mohd Hafeez, Kew, Si Na
Format: Article
Published: MDPI AG 2021
Online Access:http://psasir.upm.edu.my/id/eprint/94541/
https://www.mdpi.com/2076-3417/11/7/3117
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.upm.eprints.94541
record_format eprints
spelling my.upm.eprints.945412022-12-02T07:58:31Z http://psasir.upm.edu.my/id/eprint/94541/ Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review Nuh, Jamal Abdullahi Koh, Tieng Wei Baharom, Salmi Osman, Mohd Hafeez Kew, Si Na Many recent studies have shown that various multi-objective evolutionary algorithms have been widely applied in the field of search-based software engineering (SBSE) for optimal solutions. Most of them either focused on solving newly re-formulated problems or on proposing new approaches, while a number of studies performed reviews and comparative studies on the performance of proposed algorithms. To evaluate such performance, it is necessary to consider a number of performance metrics that play important roles during the evaluation and comparison of investigated algorithms based on their best-simulated results. While there are hundreds of performance metrics in the literature that can quantify in performing such tasks, there is a lack of systematic review conducted to provide evidence of using these performance metrics, particularly in the software engineering problem domain. In this paper, we aimed to review and quantify the type of performance metrics, number of objectives, and applied areas in software engineering that reported in primary studies—this will eventually lead to inspiring the SBSE community to further explore such approaches in depth. To perform this task, a formal systematic review protocol was applied for planning, searching, and extracting the desired elements from the studies. After considering all the relevant inclusion and exclusion criteria for the searching process, 105 relevant articles were identified from the targeted online databases as scientific evidence to answer the eight research questions. The preliminary results show that remarkable studies were reported without considering performance metrics for the purpose of algorithm evaluation. Based on the 27 performance metrics that were identified, hypervolume, inverted generational distance, generational distance, and hypercube-based diversity metrics appear to be widely adopted in most of the studies in software requirements engineering, software design, software project management, software testing, and software verification. Additionally, there are increasing interest in the community in re-formulating many objective problems with more than three objectives, yet, currently are dominated in re-formulating two to three objectives. MDPI AG 2021-03 Article PeerReviewed Nuh, Jamal Abdullahi and Koh, Tieng Wei and Baharom, Salmi and Osman, Mohd Hafeez and Kew, Si Na (2021) Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review. Applied Sciences, 11 (7). art. no. 3117. pp. 1-25. ISSN 2076-3417 https://www.mdpi.com/2076-3417/11/7/3117 10.3390/app11073117
institution Universiti Putra Malaysia
building UPM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Putra Malaysia
content_source UPM Institutional Repository
url_provider http://psasir.upm.edu.my/
description Many recent studies have shown that various multi-objective evolutionary algorithms have been widely applied in the field of search-based software engineering (SBSE) for optimal solutions. Most of them either focused on solving newly re-formulated problems or on proposing new approaches, while a number of studies performed reviews and comparative studies on the performance of proposed algorithms. To evaluate such performance, it is necessary to consider a number of performance metrics that play important roles during the evaluation and comparison of investigated algorithms based on their best-simulated results. While there are hundreds of performance metrics in the literature that can quantify in performing such tasks, there is a lack of systematic review conducted to provide evidence of using these performance metrics, particularly in the software engineering problem domain. In this paper, we aimed to review and quantify the type of performance metrics, number of objectives, and applied areas in software engineering that reported in primary studies—this will eventually lead to inspiring the SBSE community to further explore such approaches in depth. To perform this task, a formal systematic review protocol was applied for planning, searching, and extracting the desired elements from the studies. After considering all the relevant inclusion and exclusion criteria for the searching process, 105 relevant articles were identified from the targeted online databases as scientific evidence to answer the eight research questions. The preliminary results show that remarkable studies were reported without considering performance metrics for the purpose of algorithm evaluation. Based on the 27 performance metrics that were identified, hypervolume, inverted generational distance, generational distance, and hypercube-based diversity metrics appear to be widely adopted in most of the studies in software requirements engineering, software design, software project management, software testing, and software verification. Additionally, there are increasing interest in the community in re-formulating many objective problems with more than three objectives, yet, currently are dominated in re-formulating two to three objectives.
format Article
author Nuh, Jamal Abdullahi
Koh, Tieng Wei
Baharom, Salmi
Osman, Mohd Hafeez
Kew, Si Na
spellingShingle Nuh, Jamal Abdullahi
Koh, Tieng Wei
Baharom, Salmi
Osman, Mohd Hafeez
Kew, Si Na
Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
author_facet Nuh, Jamal Abdullahi
Koh, Tieng Wei
Baharom, Salmi
Osman, Mohd Hafeez
Kew, Si Na
author_sort Nuh, Jamal Abdullahi
title Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
title_short Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
title_full Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
title_fullStr Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
title_full_unstemmed Performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
title_sort performance evaluation metrics for multi-objective evolutionary algorithms in search-based software engineering: systematic literature review
publisher MDPI AG
publishDate 2021
url http://psasir.upm.edu.my/id/eprint/94541/
https://www.mdpi.com/2076-3417/11/7/3117
_version_ 1753789923246211072
score 13.211869

Similar Items