Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
token throughput
Description:
Token throughput is the rate at which a language model generates or processes text tokens during computational inference.
Primary topic:
Topic Modeling (41%)
Related topics:
Topic Modeling (41%),
Parallel Computing and Optimization Techniques (15%)
OpenAlex ID:
keywords/token-throughput
Energy use of AI inference, efficiency pathways, and test-time scaling
2026
·
Felipe Oviedo
, Fiodar Kazhamiaka
, et al.
·
Joule
·
57
Uncertainty-Aware Hybrid Inference with On-Device Small and Remote Large Language Models
2025
·
Seungeun Oh
, Jinhyuk Kim
, et al.
·
IEEE International Conference on Machine Learning for Communication and Networking (ICMLCN)
·
38
PDF
Learning Unmasking Policies for Diffusion Language Models
2025
·
Jazbec, Metod
, Olausson, Theo X.
, et al.
·
arXiv (Cornell University)
·
37
PDF
Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices
2026
·
Jie Xiao
, Qianyi Huang
, et al.
·
IEEE Transactions on Mobile Computing
·
35
PDF
LIMINAL: Exploring The Frontiers of LLM Decode Performance
2025
·
Davies, Michael
, Crago, Neal
, et al.
·
arXiv (Cornell University)
·
35
PDF
Understanding the Performance and Power of LLM Inferencing on Edge Accelerators
2025
·
Mayank Arya
, Yogesh Simmhan
·
IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum
·
25
PDF
TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers
2024
·
Kandala, Savitha Viswanadh
, Medaranga, Pramuka
, et al.
·
arXiv (Cornell University)
·
25
PDF
Hummingbird: A Smaller and Faster Large Language Model Accelerator on Embedded FPGA
2025
·
Jindong Li
, Tenglong Li
, et al.
·
18
PDF
Utility-Driven Speculative Decoding for Mixture-of-Experts
2025
·
Saxena, Anish
, Tsai, Po-An
, et al.
·
arXiv (Cornell University)
·
17
PDF
MUSTAFAR: Promoting Unstructured Sparsity for KV Cache Pruning in LLM Inference
2025
·
Donghyeon Joo
, Helya Hosseini
, et al.
·
neural information processing systems
·
15
PDF
TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling
2026
·
Junyi Chen
, Chuheng Du
, et al.
·
European Conference on Computer Systems
·
14
Speed and Conversational Large Language Models: Not All Is About Tokens per Second
2024
·
Javier Conde
, Miguel González
, et al.
·
Computer
·
14
PDF
Cost-Aware Diffusion Draft Trees for Speculative Decoding
2026
·
Zhang, Shuai
, Qiu, Huachuan
, et al.
·
arXiv (Cornell University)
·
13
PDF
GPU-to-Grid: Voltage Regulation via GPU Utilization Control
2026
·
Liang, Zhirui
, Chung, Jae-Won
, et al.
·
arXiv (Cornell University)
·
12
PDF
LEAP: LLM Inference on Scalable PIM-NoC Architecture with Balanced Dataflow and Fine-Grained Parallelism
2025
·
Yimin Wang
, Yue Jiet Chong
, et al.
·
12
PDF
Benchmarking Techniques for Real-Time Evaluation of LLMs In Production Systems
2025
·
Reena Chandra
, Rishab Bansal
, et al.
·
International Journal of Engineering Science and Information Technology
·
11
PDF
Characterizing the Behavior and Impact of KV Caching on Transformer Inferences Under Concurrency
2025
·
Jie Ye
, Jaime Cernuda
, et al.
·
IEEE International Parallel and Distributed Processing Symposium (IPDPS)
·
11
PDF
Breakthrough Low-Latency, High-Energy-Efficiency LLM Inference Performance Using NorthPole
2024
·
Rathinakumar Appuswamy
, Michael V. Debole
, et al.
·
IEEE High Performance Extreme Computing Conference
·
10
Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
2025
·
Yu, Yanpeng
, Ma, Haiyue
, et al.
·
arXiv (Cornell University)
·
9
PDF
Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission
2026
·
Seungeun Oh
, Jinhyuk Kim
, et al.
·
IEEE Transactions on Communications
·
7
PDF
ZeROf-Offload: forward-gradient scheme for efficient full parameter fine-tuning of billion-scale language models
2024
·
Jian Zhu
, Peicheng Feng
, et al.
·
Machine Learning Science and Technology
·
6
SabiYarn: Advancing Low Resource Languages with Multitask NLP Pretraining
2025
·
Jeffrey Otoibhi
, Oduguwa Damilola
, et al.
·
Workshop on African Natural Language Processing (AfricaNLP)
·
5
PDF
3D-MoE: Accelerating Multi-Expert Activated LLMs on 3D In/Near-Memory Computing Architecture via Hybrid Parallelism
2025
·
Xinyu Qu
, Zehua Zhang
, et al.
·
4
Co-design of a TinyLLM using Programmable Logic and Software on an FPGA
2024
·
Michael Muller
, Alexander Tyshka
, et al.
·
IEEE International Midwest Symposium on Circuits and Systems (MWSCAS)
·
4
UE5M3 FP4 Block Scaling for Stable Language Model Pretraining
2026
·
Hu, Robert
, Luschi, Carlo
, et al.
·
arXiv (Cornell University)
·
3
PDF
Show more works
View as search
Works count:
224
Citation count:
478
Year
Topic
Type