Work

Big Data Query Parallelization

Year: 2013
Type: dissertation
Abstract: The amount of data stored worldwide has reached the exabyte range (one quintillion of bytes).The first generation of databases based on relational algebra is hitting its limits.Therefore, companies dealing with Big Data such as Google, Facebook, or Twitter had to come up with their own frameworks and database systems in order to cope with such data scale.In parallel, a second generation of databases based on a semi-structured model like XML, YAML, or JSON rather than traditional relation databases has been emerging during the last ten years.All these systems are part of the NoSQL family which provides higher scalability and availability than traditional relational databases.In order for storage systems to scale, the data is partitioned horizontally into shards that are then replicated across several nodes.As for querying at large scales, declarative -parallelizable and highly optimizable -query languages are needed.This is what XQuery is to XML.Since the core of XQuery is free of side-effects, a huge parallelization potential lies in the execution of programs written in this language.Data-intensive queries can be distributed on a farm of computers, each of them working on a shard, while computation-intensive queries can be parallelized by duplicating sub-parts of the query plan.In this master's thesis, we investigate how execution of queries on a large amount of highly distributed data can best be orchestrated using XQuery.We come up with use cases to clearly illustrate the parallelization challenges, and present a logical model, called the Tree-Hammock pattern, that allows to parallelize XQuery programs.We also design and implement a framework which exposes parallelization capabilities to XQuery developers and allows them to manually parallelize their queries.A first version of the parallelization infrastructure is developed on top of the 28msec's platform and used to evaluate some of the use cases presented in this thesis.Experiments are then carried out to validate our approach and demonstrate performance improvements compared to sequential query processing, as well as to study which factors have the most impact on our framework.iii
Language: English
Open Access status: green
OpenAlex ID: W1603642881
Cites: 5
Cited by:
Related to: 10