Cover Image

Big data tasks execution time analysis using machine learning techniques

Big data and its analysis are in the focus of current era. The volume of data production is tremendous and a significant part of delivered data is not utilized because of the limited assets to store and process them efficiently. The world acclaimed platform that can efficiently deal with the giganti...

Full description

Saved in:
Bibliographic Details
Main Authors: Shabbir, A., Abu Bakar, K., Radzi, R. Z. R. M., Siraj, M.
Format: Conference or Workshop Item
Language:English
Published: 2019
Subjects:
QA75 Electronic computers. Computer science
Online Access:http://eprints.utm.my/id/eprint/91088/1/AishaShabbir2019_BigDataTasksExecutionTime.pdf
http://eprints.utm.my/id/eprint/91088/
http://www.ieomsociety.org/ieom2019/papers/665.pdf.
Tags: Add Tag
No Tags, Be the first to tag this record!
id my.utm.91088
record_format eprints
spelling my.utm.910882021-05-31T13:21:28Z http://eprints.utm.my/id/eprint/91088/ Big data tasks execution time analysis using machine learning techniques Shabbir, A. Abu Bakar, K. Radzi, R. Z. R. M. Siraj, M. QA75 Electronic computers. Computer science Big data and its analysis are in the focus of current era. The volume of data production is tremendous and a significant part of delivered data is not utilized because of the limited assets to store and process them efficiently. The world acclaimed platform that can efficiently deal with the gigantic amount of data in a cost effective manner is Hadoop MapReduce. In order to effectively utilize any computational platform, information about the components affecting its performance is necessary. Similarly, Hadoop MapReduce's performance can be enhanced by identifying those factors that can affect its performance. Some researchers provided some schemes for improving total task completion time of big data tasks on Hadoop MapReduce by suitable selection and scheduling of processing units i.e. mappers. However, reducers are still underexplored for its effect on the total execution time. This paper aimed at evaluation of reducer's impact on total execution time of big data tasks on Hadoop MapReduce by employing machine learning techniques. The evaluation has been carried out both analytically and experimentally by changing different number of reducers across various types and length of tasks. The results clearly depicts the dependence of total MapReduce task execution time on the number of reducers. 2019 Conference or Workshop Item PeerReviewed application/pdf en http://eprints.utm.my/id/eprint/91088/1/AishaShabbir2019_BigDataTasksExecutionTime.pdf Shabbir, A. and Abu Bakar, K. and Radzi, R. Z. R. M. and Siraj, M. (2019) Big data tasks execution time analysis using machine learning techniques. In: 9th International Conference on Industrial Engineering and Operations Management, IEOM 2019, 5-7 March 2019, JW Marriott Hotel Bangkok, Thailand. http://www.ieomsociety.org/ieom2019/papers/665.pdf.
institution Universiti Teknologi Malaysia
building UTM Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Teknologi Malaysia
content_source UTM Institutional Repository
url_provider http://eprints.utm.my/
language English
topic QA75 Electronic computers. Computer science
spellingShingle QA75 Electronic computers. Computer science
Shabbir, A.
Abu Bakar, K.
Radzi, R. Z. R. M.
Siraj, M.
Big data tasks execution time analysis using machine learning techniques
description Big data and its analysis are in the focus of current era. The volume of data production is tremendous and a significant part of delivered data is not utilized because of the limited assets to store and process them efficiently. The world acclaimed platform that can efficiently deal with the gigantic amount of data in a cost effective manner is Hadoop MapReduce. In order to effectively utilize any computational platform, information about the components affecting its performance is necessary. Similarly, Hadoop MapReduce's performance can be enhanced by identifying those factors that can affect its performance. Some researchers provided some schemes for improving total task completion time of big data tasks on Hadoop MapReduce by suitable selection and scheduling of processing units i.e. mappers. However, reducers are still underexplored for its effect on the total execution time. This paper aimed at evaluation of reducer's impact on total execution time of big data tasks on Hadoop MapReduce by employing machine learning techniques. The evaluation has been carried out both analytically and experimentally by changing different number of reducers across various types and length of tasks. The results clearly depicts the dependence of total MapReduce task execution time on the number of reducers.
format Conference or Workshop Item
author Shabbir, A.
Abu Bakar, K.
Radzi, R. Z. R. M.
Siraj, M.
author_facet Shabbir, A.
Abu Bakar, K.
Radzi, R. Z. R. M.
Siraj, M.
author_sort Shabbir, A.
title Big data tasks execution time analysis using machine learning techniques
title_short Big data tasks execution time analysis using machine learning techniques
title_full Big data tasks execution time analysis using machine learning techniques
title_fullStr Big data tasks execution time analysis using machine learning techniques
title_full_unstemmed Big data tasks execution time analysis using machine learning techniques
title_sort big data tasks execution time analysis using machine learning techniques
publishDate 2019
url http://eprints.utm.my/id/eprint/91088/1/AishaShabbir2019_BigDataTasksExecutionTime.pdf
http://eprints.utm.my/id/eprint/91088/
http://www.ieomsociety.org/ieom2019/papers/665.pdf.
_version_ 1702169643597168640
score 13.211869

Similar Items