A Data Science Platform for the National Oceanography Centre and Beyond
Year: 2025
Type: article
Abstract: One of the common issues faced in the world of research software is "it worked on my machine". In this talk, we will present our efforts at the National Oceanography Centre (NOC) to solve this problem and make the task of writing shareable and reusable software easier by building an interactive Data Science Platform (DSP) based upon JupyterHub. The platform is accessed through a web browser and includes the Pangeo [https://pangeo.io/] stack along with popular machine learning libraries. In addition to a Jupyter notebooks interface we have integrated Matlab and a remote desktop service, removing the need for users to install these locally or rely on X forwarding/NoMachine/VNC. The DSP was first deployed on servers owned by NOC. We are employing cloud native technologies such as Kubernetes and S3 style object stores as the basis for much of our digital research infrastructure. This allows us to deploy instances of the DSP on commercial cloud and national computing facilities, providing a uniform user experience regardless of the underlying system. Where the DSP’s server is not powerful enough, we use Dask to enable access to high performance computing or cloud computing resources. In addition to providing a resource for developing and running research code, the platform is ideal for training. We use it regularly for Carpentries workshops since it provides all the software the learners require, resulting in a significant time saving during workshops, and allows ongoing access after the workshop. As well as a demonstration of this platform, we will present our experiences of deploying and running it, including lessons we have learnt along the way and our plans for its future. Acknowledgements This work has been funded by the Natural Environment Research Council (NERC) projects Atlantic Climate and Environment Strategic Science (AtlantiS) and Climate Linked Atlantic Sector Science (CLASS).A recording of this session is available on YouTube: https://youtu.be/ovKPLFRQwdw
Authors: Sauze, Colin, Turner, Esther
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7115941307
FWCI:
Cites:
Cited by:
Related to: