Work

Data Wrangling for Big Data: Challenges and Opportunities

Year: 2016

Type: conference-paper

Abstract: Data wrangling is the process by which the data required by an application is identified, extracted, cleaned and integrated, to yield adata set that is suitable for exploration and analysis. Although there are widely used Extract, Transform and Load (ETL) techniques and platforms, they often require manual work from technical and domain experts at different stages of the process. When confronted with the 4 V’s of big data (volume, velocity, variety and veracity),manual intervention may make ETL prohibitively expensive. This paper argues that providing cost-effective, highly-automated approaches to data wrangling involves significant research challenges,requiring fundamental changes to established areas such as data extraction,integration and cleaning, and to the ways in which these areas are brought together. Specifically, the paper discusses the importance of comprehensive support for context awareness within data wrangling, and the need for adaptive, pay-as-you-go solutions that automatically tune the wrangling process to the requirements and resources of the specific application.

Source: Research Explorer (The University of Manchester)

DOI: 10.5441/002/edbt.2016.44

Authors: Furche, Tim, Gottlob, Georg, Libkin, Leonid, Orsi, Giorgio, Paton, Norman

Institutions: University of Oxford, University of Birmingham, University of Manchester

Language: English

Topic: Data Quality and Management

Subfield: Management Science and Operations Research

Field: Decision Sciences

Domain: Social Sciences

Open Access status: green

Funder: Engineering and Physical Sciences Research Council

OpenAlex ID: W2493970676

FWCI: 13.94

Cites: 34

Cited by: 73

Related to: 10