Product
Help
Pricing
About
Jobs
Log in
Get started
Keyword
Add to collection
request batching
Description:
A computing technique that groups multiple requests for processing or transmission together to improve efficiency or handle high demand.
Related topics:
Cloud Computing and Resource Management (14%),
Caching and Content Delivery (12%),
Parallel Computing and Optimization Techniques (11%)
OpenAlex ID:
keywords/request-batching
Efficient Memory Management for Large Language Model Serving with PagedAttention
2023
·
Woosuk Kwon
, Zhuohan Li
, et al.
·
ACM SIGOPS Symposium on Operating Systems Principles (SOSP)
·
10,855
PDF
Clipper: A Low-Latency Online Prediction Serving System
2016
·
Crankshaw, Daniel
, Wang, Xin
, et al.
·
arXiv (Cornell University)
·
683
PDF
Scheduling policies for an on-demand video server with batching
1994
·
A. Dan
, D. Sitaram
, et al.
·
ACM International Conference on Multimedia (ACM MM)
·
626
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
2024
·
Agrawal, Amey
, Kedia, Nitin
, et al.
·
arXiv (Cornell University)
·
539
PDF
MArk: Exploiting Cloud Services for Cost-Effective, SLO-Aware Machine Learning Inference Serving.
2019
·
Chengliang Zhang
, Minchen Yu
, et al.
·
Rare & Special e-Zone (The Hong Kong University of Science and Technology)
·
413
Dynamic batching policies for an on-demand video server
1996
·
Asit Dan
, Dinkar Sitaram
, et al.
·
Multimedia Systems
·
402
On optimal batching policies for video-on-demand storage servers
2002
·
C.C. Aggarwal
, J.L. Wolf
, et al.
·
IEEE International Conference on Multimedia Computing and Systems (ICMCS)
·
244
Energy conservation policies for web servers
2003
·
Mootaz Elnozahy
, Michael Kistler
, et al.
·
USENIX Symposium on Internet Technologies and Systems
·
227
Atom: Low-bit Quantization for Efficient and Accurate LLM Serving
2023
·
Zhao, Yilong
, Lin, Chien-Yu
, et al.
·
arXiv (Cornell University)
·
193
PDF
Performance model of interactive video-on-demand systems
1996
·
V.O.K. Li
, Wanjiun Liao
, et al.
·
IEEE Journal on Selected Areas in Communications
·
133
Broadcast on demand: efficient and timely dissemination of data in mobile environments
2002
·
P. Xuan
, S. Sen
, et al.
·
IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS)
·
132
Channel Allocation under Batching and VCR Control in Video-on-Demand Systems
1995
·
A. Dan
, P. Shahabuddin
, et al.
·
Journal of Parallel and Distributed Computing
·
122
Distributed servers architecture for networked video services
2001
·
S.H.G. Chan
, F. Tobagi
·
IEEE/ACM Transactions on Networking
·
121
FIRM: Fair and High-Performance Memory Control for Persistent Memory Systems
2014
·
Jishen Zhao
, Onur Mutlu
, et al.
·
IEEE/ACM International Symposium on Microarchitecture (MICRO)
·
120
Cooperative I/O: a novel I/O semantics for energy-aware applications
2002
·
Andreas Weissel
, Björn Beutel
, et al.
·
ACM SIGOPS Operating Systems Review
·
116
Optimizing inference serving on serverless platforms
2022
·
Ahsan Ali
, Riccardo Pinciroli
, et al.
·
Proceedings of the VLDB Endowment
·
89
Multicast with cache (Mcache): an adaptive zero-delay video-on-demand service
2002
·
S. Ramesh
, Injong Rhee
, et al.
·
IEEE Conference on Computer Communications (INFOCOM)
·
84
TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput
2024
·
Liu, Xiaoxuan
, Park, Jongseok
, et al.
·
arXiv (Cornell University)
·
80
PDF
Multicast with cache (Mcache): an adaptive zero-delay video-on-demand service
2001
·
S. Ramesh
, I. Rhee
, et al.
·
IEEE Transactions on Circuits and Systems for Video Technology
·
76
The maximum factor queue length batching scheme for video-on-demand systems
2001
·
C.C. Aggarwal
, J.L. Wolf
, et al.
·
IEEE Transactions on Computers
·
73
Continuous Media Sharing in Multimedia Database Systems
1995
·
M. Kamath
, Krithi Ramamritham
, et al.
·
Database Systems for Advanced Applications
·
73
Scheduling Page Table Walks for Irregular GPU Applications
2018
·
Seunghee Shin
, Guilherme Cox
, et al.
·
Proceedings - International Symposium on Computer Architecture/Proceedings
·
70
Enabling Cost-Effective, SLO-Aware Machine Learning Inference Serving on Public Cloud
2020
·
Chengliang Zhang
, Minchen Yu
, et al.
·
IEEE Transactions on Cloud Computing
·
64
NEO: Saving GPU Memory Crisis with CPU Offloading for Online LLM Inference
2024
·
Jiang, Xuanlin
, Zhou, Yang
, et al.
·
arXiv (Cornell University)
·
62
PDF
Online Service with Delay
2017
·
Yossi Azar
, Arun Ganesh
, et al.
·
ACM Transactions on Algorithms
·
56
PDF
Show more works
View as search
Works count:
217
Citation count:
16,900
Year
Topic
Type