Work

Scalable Transactional Stream Processing on Multicore Processors

Year: 2025

Type: article

Abstract: Transactional stream processing engines (TSPEs) are central to modern stream applications handling shared mutable states. However, their full potential, particularly in adaptive scheduling, remains largely unexplored. We presentMorphStream, a TSPE designed to optimize parallelism and performance for transactional stream processing on multicores. Through a unique three-stage execution paradigm (i.e.,planning,scheduling, andexecution),MorphStreamenables adaptive scheduling under varying workload characteristics. Building on this foundation,MorphStreamis further enhanced with support for non-deterministic state access, employing a stateful task precedence graph to handle undefined read/write sets at runtime while guaranteeing transaction semantics. Additionally,MorphStreamincorporates a generalized framework for managing window-based operations, enabling efficient tracking and maintenance of overlapping windows using multi-versioned state management. These extensions enhance the system's ability to process dynamic and irregular workloads. Experimental results demonstrate up to 3.4 times higher throughput and 69.1% lower latency compared to state-of-the-art TSPEs, validating its scalability and adaptability in real-world streaming scenarios.

Source: IEEE Transactions on Knowledge and Data Engineering

DOI: 10.1109/tkde.2025.3556741

Authors: Jianjun Zhao, Yancan Mao, Zhonghao Yang, Haikun Liu, Shuhao Zhang

Institutions: Huazhong University of Science and Technology, National University of Singapore, Nanyang Technological University

Language: English

Topic: Advanced Database Systems and Queries

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,645

Funder: National Natural Science Foundation of China