Year: 2022
Type: conference-paper
Abstract: String similarity search and join are essential op-erations in many fields. Existing solutions adopt a filter-and-verification framework and build inverted indexes based on generated signatures to prune dissimilar candidates. While existing solutions mainly focus on improving the query processing performance, little attention is paid to reducing the inverted indexes' memory consumption. In cases where the index size is larger than the memory, users have to employ more expensive disk-based algorithms rather than in-memory ones. In this paper, we propose a flexible framework CSS to reduce the index size and keep high query performance for string search and join applications. It can be easily incorporated into a broad scope of existing frameworks. We first give improved solutions for offline inverted lists construction to better support string similarity search. Nevertheless, they cannot be applied in the problem of string similarity join where indexes are constructed online. To address this issue, we further propose the first approach for online construction of compressed inverted lists. We theoretically study a benefit model to help find the best trade-off between memory consumption and execution time, and then propose an adaptive compression approach based on it. Experimental results on large-scale datasets demonstrate that CSS can reduce the memory consumption by 3 to 5 times while having similar or even better query processing performance for a variety of string similarity search and join frameworks.
Authors: Guorui Xiao, Jin Wang, Chunbin Lin, Carlo Zaniolo
Institutions: University of California, Los Angeles, Amazon (United States)
Language: English
Subfield: Theoretical Computer Science
Field: Mathematics
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W4289533992