UDMem: Scalable Memory Management for Massively Parallel Distributed Memory Architecture
Year: 2026
Type: dissertation
Abstract: In recent decades, emerging high-performance big data applications have scaled by orders of magnitude in both data and computation. To overcome single-node systems' limits, modern architectures aggregate hundreds to thousands of compute and memory nodes into distributed global memory systems. As system memory scales to petabytes, memory management becomes increasingly challenging: in common distributed programming models like Message Passing Interface (MPI) and PGAS, programmers often fall back to manual data management across distinct address spaces, which compromises programmability and machine portability. We believe global memory management and virtualization are essential for programming efficiency, software complexity, and application flexibility in distributed systems. Yet, existing solutions fall short. Traditional page translation cannot scale because the page table grows linearly as the data exceeds the TLB reach, whereas segmentation fails to exploit data locality or distributed memory bandwidth. Conventional memory management prioritizes generality. However, modern applications have drastically different memory allocation patterns, ranging from monolithic, one-time allocations to high-frequency, fine-grained requests, creating severe performance bottlenecks for global memory managers. A system-wide unified allocator cannot handle this demand variation efficiently. To address these challenges, we propose UDMem, a global memory management scheme tailored for massively parallel, distributed-memory environments. UDMem consists of two components: UDTrans: A compact, scalable, segment-based address translation scheme for scale-out distributed memories. It delivers high-performance translation with negligible overhead while exposing programmer-controlled data layouts across memories for data locality. GLMalloc: A high-throughput, hierarchical memory management scheme, combining a locality-aware, privileged global allocator with a throughput-optimized user-level allocator for highly skewed, fine-grained allocations. We evaluate UDMem across comprehensive trace-driven and cycle-accurate simulations and show that it eliminates address-translation DRAM traffic amplification with >99.99% TLB hit rates, delivers atomic-free fast-path allocations as long as 77 cycles while sustaining over 500 billion allocation operations per second and improves end-to-end application performance by up to 1.49x via locality-aware hardware-software co-design. In summary, UDMem provides high-throughput and energy-efficient global memory management across various applications, promoting programmability and portability for future distributed global-memory systems.
Source: University of Chicago
DOI: 10.6082/2k2aq-vb232
Author: Wang, Yuqing
Institution: University of Chicago
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7222299858
