Year: 2025
Type: dissertation
Abstract: Modern scientific applications in domains such as deep learning, data analytics, signal processing, and genomics generate vast amounts of data and demand high computational performance. The advent of advanced Artificial Intelligence (AI) models, particularly transformer-based Large Language Models (LLMs), has intensified these demands by processing enormous datasets for tasks such as text summarization and translation. These data-intensive workloads have exposed significant limitations in traditional computing architectures, notably the widening gap between processor speeds and memory access latency, referred to as the memory wall, and the constraints imposed by bandwidth limitations, known as the bandwidth wall. Despite advancements following Moore's Law and Dennard's scaling, recent years have seen limited single-thread performance improvements and decreased returns from adding more cores due to memory subsystem constraints. This has motivated the exploration of alternative architectural solutions to meet the increasing demands of modern applications. This dissertation addresses these critical challenges by investigating and advancing the field of Near-Memory Computing (NMC), with a particular focus on Field-Programmable Gate Array (FPGA)-based systems integrated with High Bandwidth Memory (HBM). NMC shifts the computational paradigm by bringing processing capabilities closer to where data resides, thereby reducing the costly data movement between memory and processors and reducing the bottlenecks associated with the memory and bandwidth walls. The goals of this dissertation are to provide practical methodologies and tools that facilitate the adoption of FPGA-based NMC systems. Specifically, this work contributes novel solutions in NMC simulation frameworks, automatic kernel characterization techniques, generalized compilation approaches for legacy code, and comprehensive optimization frameworks for workload mapping. Fast and accurate simulation tools are essential for early-stage design space exploration, especially when working with complex memory hierarchies like those in HBM-enabled FPGA systems. They allow architects to evaluate performance trade-offs and guide design decisions without requiring full hardware implementation. In this context, a simulation framework was developed to model HBM2-based FPGA platforms with improved accuracy. The simulator models the HBM2 memory and the specific interconnect modules used in AMD/Xilinx devices. This tool fills a gap in existing prototyping methods, enabling practical system-level design space exploration and evaluation of NMC architectures using real-world applications. By accurately modeling the interactions between processing elements and the memory subsystem, the simulator aids architects in optimizing NMC systems to effectively handle data-intensive workloads. Once the hardware platform specifications, such as memory hierarchy, bandwidth, and interconnect structure, are known, performance models can be constructed to estimate how different application kernels will behave on that specific architecture. To this end, this dissertation presents application characterization and performance prediction methodologies. Machine learning models, specifically random forest algorithms, were employed to estimate the performance of application kernels on HBM and Hybrid Memory Cube (HMC)-based systems. Additionally, a compiler-based framework was developed to assist developers in selecting NMC kernels without requiring extensive hardware knowledge or profiling of unknown kernels. After NMC candidate kernels are identified, it is essential that there exists a methodology to map user legacy kernel code to NMC accelerators. In this direction, the thesis focuses on compiling linear algebra kernels to FPGA-HBM accelerators using a framework comprising both software and hardware components. The software component introduces a toolflow for detecting Matrix-Vector Multiplication (MVM) and matrix multiplication kernels in user code and replacing MVM kernels with calls to an FPGA accelerator. On the hardware side, a novel FPGA architecture was designed to efficiently handle both Dense Matrix-Vector Multiplication (GEMV) and Sparse Matrix-Vector Multiplication (SpMV) operations. By leveraging the high memory bandwidth of HBM, the architecture overcomes memory bottlenecks, providing a robust solution for accelerating matrix-vector multiplication on FPGA. Evaluation results demonstrate speedups and increased bandwidth efficiency compared to state-of-the-art solutions on CPUs, Graphical Processing Units (GPUs), and other FPGAs. To support workload scalability and concurrent application execution on NMC platforms, this dissertation introduces an optimization framework for efficiently mapping multi-kernel and multi-application workloads onto FPGA-HBM platforms. By formulating a system-level optimization problem, the framework recommends optimal parallelization strategies and effective mappings of interfaces to HBM channels. To explore the extensive design space, meta-heuristic algorithms, including Genetic Algorithms, Particle Swarm Optimization, and Differential Evolution, are employed. Particle Swarm Optimization achieves the best balance between exploration efficiency and computational speed. Additionally, a machine learning-based resource estimator predicts the utilization of critical FPGA resources, enabling rapid design exploration without the need for full hardware synthesis. The framework is validated on an FPGA platform using Convolutional Neural Network (CNN) kernels and multiple concurrent application workloads, demonstrating enhanced performance and adaptability to heterogeneous applications. This work effectively prioritizes tasks based on user-defined criteria and allocates resources to meet bandwidth and scalability requirements. The proposed tools and frameworks facilitate the adoption of NMC paradigms by providing practical methods for accurate modeling, intelligent application characterization, novel accelerator design, and optimized workload mapping methods. These algorithms contribute to the design of efficient NMC architectures capable of meeting the demands of modern computational workloads. Ultimately, this work lays a foundation for developing high-performance computing architectures that effectively leverage near-memory computing to address the limitations of traditional systems.
Author: Iskandar, Veronia
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Keywords: near-memory computing, 3D stacked memory, high-bandwidth memory, FPGA, HBM2, memory wall, kernel characterization, workload mapping, performance prediction, random forest, large language models, deep learning workloads, design space exploration, interconnect modeling, memory latency, data-intensive workloads
Open Access status: green
OpenAlex ID: W7133274745
