Work

Prediction-based auto-scaling of scientific workflows

Year: 2011

Type: conference-paper

Abstract: In this paper we propose a novel method for auto-scaling data-centric workflow tasks. Scaling is achieved through a prediction mechanism where the input data load on each task within a workflow is used to compute the estimated task execution time. Through load prediction, the framework can take informed decisions on scaling multiple workflow tasks independently to improve overall throughput and reduce workflow bottlenecks. This method was implemented in the WS-VLAM workflow system and with an image analyses workflow we show that this technique achieves faster data processing rates and reduces overall workflow makespan.

Source: Middleware for Grid Computing

DOI: 10.1145/2089002.2089003

Authors: Reginald Cushing, Spiros Koulouzis, Adam S. Z. Belloum, Marian Bubak

Institutions: University of Amsterdam, AGH University of Krakow

Language: English

Topic: Distributed and Parallel Computing Systems

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: green

OpenAlex ID: W2027541616

FWCI: 1.563

Cites: 13

Cited by: 14

Related to: 10