Year: 2024
Type: conference-paper
Abstract: Modern Function-as-a-Service (FaaS) cloud platforms offer great potential for supporting event-driven scientific workflows. Nonetheless, there remain barriers to adoption by the scientific community in domains such as environmental sciences, where R is the focal language used for the development of applications and where users are typically not well-versed with FaaS APIs. This paper describes the design and implementation of FaaSr, a novel middleware system that supports event-driven scientific workflows in R. A key novelty in FaaSr is the ability to deploy workflows across FaaS providers without the need for any managed servers for coordination. With FaaSr: 1) functions are written in R; 2) the runtime environments for their execution are customizable containers; 3) functions access data in cloud storage (S3) with a familiar file-based abstraction supporting both full file put/get primitives and subsetting using the Parquet format; and 4) function invocation and workflow coordination only requires S3 cloud object storage, without relying on any dedicated, active workflow engine server or cloud-specific queues/databases. The paper reports on the functionality and performance of FaaSr for micro-benchmarks and two case studies: event-driven forecast and batch job workflows. These demonstrate the ability to deploy workflows across multiple platforms (GitHub Actions, Amazon Web Services Lambda, and the open-source OpenWhisk), without the need for dedicated coordination servers, across both cloud and edge resources. FaaSr is open-source and available as a CRAN package.
Institutions: University of Florida, Virginia Tech, Oregon State University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Awards: G1453181110, Track 4: Advanced CI Coordination Ecosystem: Monitoring and Measurement Services, Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R, G3562386289, Collaborative Research: Elements: FaaSr: Enabling Cloud-native Event-driven Function-as-a-Service Computing Workflows in R, Track 1: ACCESS Resource Allocations Marketplace and Platform Services (RAMPS), G4010157413, ACO: An Open CI Ecosystem to Advance Scientific Discovery (OpenCI), Track 3: COre National Ecosystem for CyberinfrasTructure (CONECT), G5367127336, G5464919783, Collaborative Research: CIBR: Cyberinfrastructure Enabling End-to-End Workflows for Aquatic Ecosystem Forecasting, Collaborative Research: URoL:ASC: Applying rules of life to forecast emergent behavior of phytoplankton and advance water quality management, Collaborative Research: URoL:ASC: Applying rules of life to forecast emergent behavior of phytoplankton and advance water quality management, Collaborative Research: CIBR: Cyberinfrastructure Enabling End-to-End Workflows for Aquatic Ecosystem Forecasting, Track 2: Customized Multi-tier Assistance, Training, and Computational Help (MATCH) for End User ACCESS to CI
OpenAlex ID: W4402673978