Year: 2015
Type: preprint
Abstract: Recent efforts in data cleaning of structured data have focused exclusively on problems like data deduplication, record matching, and data standardization; none of the approaches addressing these problems focus on fixing incorrect attribute values in tuples. Correcting values in tuples is typically performed by a minimum cost repair of tuples that violate static constraints like CFDs (which have to be provided by domain experts, or learned from a clean sample of the database). In this paper, we provide a method for correcting individual attribute values in a structured database using a Bayesian generative model and a statistical error model learned from the noisy database directly. We thus avoid the necessity for a domain expert or clean master data. We also show how to efficiently perform consistent query answering using this model over a dirty database, in case write permissions to the database are unavailable. We evaluate our methods over both synthetic and real data.
Source: arXiv (Cornell University)
Authors: De, Sushovan, Hu, Yuheng, Vamsikrishna, Meduri Venkata, Chen, Yi, Kambhampati, Subbarao
Institution: Ford Motor Company (United States)
Language: English
Topic: Data Quality and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W774107361