Work

SkyServe: Serving AI Models across Regions and Clouds with Spot Instances

Year: 2025

Type: conference-paper

Abstract: Recent years have witnessed an explosive growth of AI models. The high cost of hosting AI services on GPUs and their demanding service requirements, make it timely and challenging to lower service costs and guarantee service quality. While spot instances have long been offered with a large discount, spot preemptions have discouraged users from using them to host model replicas when serving AI models.

DOI: 10.1145/3689031.3717459

Authors: Ziming Mao, Tian Xia, Zhanghao Wu, Wei-Lin Chiang, Tyler Griggs, Romil Bhardwaj, Zongheng Yang, Scott Shenker, Ion Stoica

Institutions: University of California, Berkeley, International Computer Science Institute

Language: English

Topic: Cloud Computing and Resource Management

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

OpenAlex ID: W4408846413

FWCI: 18.8

Cites: 21

Cited by: 16

Related to: 10