Work

Enabling Efficient MLOps in Weather and Climate Services at MeteoSwiss

Year: 2025
Type: conference-abstract
Abstract: As machine learning (ML) matures from experimentation to operational deployment, institutions face the challenge of integrating ML into their production environments in a sustainable, scalable, and collaborative manner. This contribution shares the ongoing approach taken by MeteoSwiss to develop a cross-cutting ML capability that supports domain-specific innovation while laying the groundwork for robust operational practices. To this end, we are building an MLOps foundation that integrates with our existing DevOps culture and tools, enabling agile development, cross-institutional collaboration, and long-term operational reliability. Central to this effort is the design of an evolving ML platform that aligns with open standards and prioritizes reproducibility, monitoring, and modularity. The development workflows we support are inherently complex, involving multi-source data pipelines, extended model training sessions, and inference jobs for regular predictions using new data. This complexity is compounded by a fragmented infrastructure landscape spanning high-performance computing (HPC) and cloud environments—national (CSCS), international (European Weather Cloud operated by ECMWF and EUMETSAT), and private (AWS) providers. We present the architectural principles guiding both platform and process development, including CI/CD for ML pipelines, infrastructure-as-code, testing of data and models, metadata management, and model deployment strategies. Design decisions—such as the adoption of open-source tools for orchestration and lifecycle management—are discussed in the context of public sector constraints, such as limited resources and the need for transparency and auditability. While the platform is under active development, we illustrate its current capabilities through a concrete use case, and reflect on the processes that support the sustainable evolution of the ML operations and collaboration at MeteoSwiss. This contribution aims to foster discussion around best practices for implementing MLOps in public institutions, the role of cloud ecosystems in operational ML, and how to architect systems that are technically robust and open to collaboration across the weather and climate community.
Language: English
Open Access status: gold
OpenAlex ID: W4411805006
Cites:
Cited by:
Related to: 9