Indexing strategy for big data processing: A case study of PingER
With the huge amount of data continuously accumulated and shared by individuals and organizations, it has become necessary to meet the emerging processing and retrieval requirements associated with these large volumes of complex data. This could be achieved by indexing the data sets and reducing he...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English English |
| Published: |
2015
|
| Subjects: | |
| Online Access: | http://etd.uum.edu.my/5604/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | With the huge amount of data continuously accumulated and shared by individuals and organizations, it has become necessary to meet the emerging processing and retrieval
requirements associated with these large volumes of complex data. This could be achieved by indexing the data sets and reducing heavy computational overhead accustomed to most current indexing strategies during processing of very large amount of data sets. This study proposed a novel Indexing strategy called Big Data INDexing Strategy (BIND), using a concept of high performance parallel computing. BIND supports parallel distribution of data and performs processing in a MapReduce fashion. To
develop BIND strategy, Ian foster’s task-scheduling concept for parallel processing is applied. The proposed indexing strategy was first tested on a 2-node cluster environment
where varying sizes of datasets were used to note if the performance improves or declines as the size of the data increases. Subsequently, it was tested on a 3-node cluster to note the performance when the number of computation resources are increased. The results demonstrate that BIND minimizes the processing and query time as compared to the current strategy. The findings have significant implication in efficiently managing Big Data and facilitating data storage and information retrieval for users and organizations that manage Big Data. |
|---|