Work

Exploring Function Granularity for Serverless Machine Learning Application with GPU Sharing

Year: 2025

Type: article

Abstract: Recent years have witnessed increasing interest in machine learning (ML) inferences on serverless computing due to its auto-scaling and cost-effective properties. However, one critical aspect, function granularity, has been largely overlooked, limiting the potential of serverless ML. This paper explores the impact of function granularity on serverless ML, revealing its important effects on the SLO hit rates and resource costs of serverless applications. It further proposes adaptive granularity as an approach to addressing the phenomenon that no single granularity fits all applications and situations. It explores three predictive models and presents programming tools and runtime extensions to facilitate the integration of adaptive granularity into existing serverless platforms. Experiments show adaptive granularity produces up to a 29.2% improvement in SLO hit rates and up to a 24.6% reduction in resource costs over the state-of-the-art serverless ML which uses fixed granularity.

Source: ACM SIGMETRICS Performance Evaluation Review

DOI: 10.1145/3744970.3727285

Authors: Xinning Hui, Yuanchao Xu, Xipeng Shen

Institutions: North Carolina State University, University of California, Santa Cruz

Language: English

Topic: Cloud Computing and Resource Management

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: bronze

Funders: National Science Foundation, Universitas Brawijaya

Awards: Collaborative Research: CSR: Medium: Scaling Secure Serverless Computing on Heterogeneous Datacenters, G6777333718