Year: 2025
Type: article
Abstract: Data processing frameworks like Apache Spark and Flink provide built-in support for user-defined aggregation functions (UDAFs), enabling the integration of domain-specific logic. However, for these frameworks to support efficient UDAF execution, the function needs to satisfy a homomorphism property, which ensures that partial results from independent computations can be merged correctly Motivated by this problem, this paper introduces a novel homomorphism calculus that can both verify and refute whether a UDAF is a dataframe homomorphism. If so, our calculus also enables the construction of a corresponding merge operator which can be used for incremental computation and parallel execution. We have implemented an algorithm based on our proposed calculus and evaluate it on real-world UDAFs, demonstrating that our approach significantly outperforms two leading synthesizers.
DOI: 10.1145/3763072
Institution: The University of Texas at Austin
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
Awards: SHF: Medium: Collaborative Research: Bridging Automated Formal Reasoning and Continuous Optimization for Provably Safe Deep Learning, Collaborative Research: SaTC: CORE: Large: Building and Deploying a Verified JavaScript Runtime, FMitF: Track I: Program Synthesis for Robot Learning from Demonstrations, FMitF: Track I: Performance Verification for Networked Systems, SHF: Medium: Collaborative Research: Computer-Aided Programming for Data Science, SaTC: CORE: Medium: Collaborative: Effective Formal Reasoning for Mobile Malware, SHF: Medium: Neurosymbolic Agents for Formal Theorem-Proving, Expeditions: Collaborative Research: Understanding the World Through Code, G7481599941, Collaborative Research: SHF: Core: Medium: Program Synthesis for Schema Changes
OpenAlex ID: W4415005576