Distributed compressive genomics: Fundamental pattern matching primitives via spark
Year: 2025
Type: article
Abstract: • We develop distributed FM-Index and CBM algorithms for scalable compressed pattern matching on large genomic collections using Apache Spark. • Weconductathoroughperformanceevaluationbasedonstandardbenchmarksandmetricsusedindistributedalgorithm analysis. • We provide a publicly available software library to easily integrate distributed compressed pattern matching into genomic data processing pipelines without requiring a deep distributed programming expertise. Compressive genomics leverages compressed data representations to enhance the efficiency of bioinformatics tasks like sequence comparison and search. Surprisingly, the fundamental operation of pattern matching on large DNA sequence collections remains unexplored in the realm of genomic analysis. However, distributed systems like Spark offer the scalability necessary to process increasingly large genomic datasets efficiently. We present the first Spark-based implementation of the FM-Index and Compressed Boyer-Moore (CBM) algorithms, evaluating their performance and providing insights into their advantages for large-scale bioinformatics applications. A comprehensive experimental study demonstrates clear performance gains over uncompressed approaches. Furthermore, we introduce SparkGeco , a distributed compressive genomics software library designed to simplify the integration of FM-Index and CBM algorithms into DNA sequence analysis pipelines within Apache Spark, thus supporting the development of efficient and scalable genomic analysis workflows. This work provides a concrete step towards high-performance, data-centric eScience solutions in computational biology.
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,000
Awards: G5636153502, G7103356121, G7205771190
OpenAlex ID: W4414606495