Using machine learning for intelligent shard sizing on the cloud
Year: 2025
Type: article
Abstract: Sharding implementations use conservative approximations for determiningthe number of cloud instances required and the size of the shards to be storedon each of them. Conservative approximations are often inaccurate and resultin overloaded deployments, which need reactive refinement. Reactiverefinement results in demand for additional resources from an alreadyoverloaded system and is counterproductive.This paper proposes an algorithm that eliminates the need for conservativeapproximations and reduces the need for reactive refinement. A multiplelinear regression based machine learning algorithm is used to predict thelatency of requests for a given application deployed on a cloud machine. Thepredicted latency helps to decide accurately and with certainty if the capacityof the cloud machine will satisfy the service level agreement for effectiveoperation of the application. Application of the proposed methods on apopular database schema on the cloud resulted in highly accurate predictions.The results of the deployment and the tests performed to establish theaccuracy have been presented in detail and are shown to establish theauthenticity of the claims.
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7117698613
FWCI:
Cites:
Cited by:
Related to: