A Light-Speed Large Language Model Accelerator with Optical Stochastic Computing
Year: 2025
Type: conference-paper
Abstract: To address the increasingly intensive computational demands of attention-based large language models (LLMs), there is a growing interest in developing energy-efficient and high-speed hardware accelerators. To that end, photonics is being considered as an alternative technology to digital electronics. This work introduces a novel optical hardware accelerator that leverages stochastic computing principles for LLMs. Our proposed accelerator incorporates full-range optical stochastic multipliers and stochastic-analog compute-capable optical-to-electrical transducer units to efficiently handle static and dynamic tensor computations in attention-based models. Our analysis shows that our accelerator exhibits at least 7.6× speedup and 1.3× lower energy compared to state-of-the-art LLMs hardware accelerators.
Institutions: Colorado State University, University of Kentucky
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4411713541