Year: 2025
Type: article
Abstract: . The Earth System Grid Federation (ESGF) holds several petabytes of climate data distributed across millions of files held in data centers worldwide. Obtaining and manipulating the scientific information (climate variables) held in these files is non-trivial. The ESGF Virtual Aggregation is one of several solutions to providing an out-of-the-box aggregated and analysis ready view of those variables. Here we discuss the ESGF Virtual Aggregation in the context of the existing infrastructure, and some of those other solutions providing analysis ready data. We describe how it is constructed, how it can be used, and provide some performance evaluation. It will be seen that the ESGF Virtual Aggregation provides a sustainable solution to some of the problems encountered in producing analysis ready data, without the cost of data replication to different formats, albeit at the cost of more data movement within the analysis than some alternatives. If heavily used, it may also require more ESGF data servers than are currently deployed in data node deployments. The need for such data servers should be a component of ongoing discussions about the future of the ESGF and its constituent core services.
Source: Geoscientific model development
Language: English
Subfield: Information Systems and Management
Field: Decision Sciences
Domain: Social Sciences
SDG: Climate action
Open Access status: gold
Estimated APC paid: $1,725
Awards: G1175115578, G121809054, G1599646647, G1703969385, G1818268936, G1855990336, G2009339637, Infrastructure for the European Network for Earth System modelling - Phase 3, G2262748287, G2309174360, G2381735332, G2601397997, G2660406951, G2742324050, G2745972340, G300979063, G3480869486, G3518538256, G3681454997, G3690828678, G4042783231, G425668148, G430777504, G434337789, G4442677895, G4526231891, G5080475149, G5195188938, G5224080922, G5421974903, G5450994985, G5805502524, IMPETUS4CHANGE (I4C): IMPROVING NEAR-TERM CLIMATE PREDICTIONS FOR SOCIETAL TRANSFORMATION, G6218905885, G6295457465, G6550679324, G6978662411, G7084143925, G8684520763, G8888022036, G99296377
OpenAlex ID: W4409962736