When good models aren’t enough
For decades, Germany has been one of the world’s leading centers for climate and Earth system modeling. Models for the atmosphere, ocean, land surface, atmospheric chemistry, biogeochemistry, and ice sheets are developed at numerous universities and research institutions. Many of these models are highly regarded internationally and make important contributions to research, climate reports, forecasts, and projections of future environmental changes.
This modeling landscape has evolved over decades. Various institutions have developed their own models, tools, and development practices — tailored to their respective scientific priorities. While this historically accumulated scientific expertise is a major strength, it also makes collaboration across institutional boundaries difficult today. As a result, the same solutions are developed multiple times, knowledge remains confined to individual groups, and new researchers are often confronted with a complex landscape of numerous models and software tools addressing the same scientific questions and technical requirements.
At the same time, many modeling groups face the challenge of adapting historically developed code to modern high-performance computers, new data infrastructures, and future computing architectures in order to remain internationally competitive in the future.
The question is therefore increasingly: How can these historically diverse research groups work together on the future of Earth system and climate modeling? How do we bring this diverse expertise together and make efficient use of our technical, human, and financial resources? And how can we sustain these efforts in the long term?
This is precisely where natESM comes in. Together with partners from the German Earth system modeling community, the project — coordinated by the DKRZ — develops and operates structures that support open development, sustainable scientific software, and the efficient use of modern high-performance computing systems.
Diversity as a strength and a challenge
The national Earth system modeling strategy natESM therefore aims to strengthen collaboration across the German Earth system modeling community and facilitate access to collaboratively developed software. The goal is not to formulate or even dictate scientific priorities. On the contrary: natESM views the disciplinary diversity of the numerous German modeling groups as one of the research community’s greatest strengths, which must now be further developed into a collectively managed ESM system.
In recent years, we have laid important groundwork for this and defined the first components of this complex natESM system. In addition to the core components for modeling the atmosphere, ocean, and land1, which form the basis of the natESM system, there are models that extend these core components to include additional processes, such as atmospheric chemistry, biogeochemistry, or ice sheet dynamics (so-called “extended-core components”2). Our infrastructure components3 enable the integration of the various component models depending on the scientific question at hand.
At the same time, numerous other models are awaiting further technical and scientific development so they can become part of the natESM system in the future. But how do we arrive at a system capable of answering the many diverse questions posed by the Earth system and climate research community?
Figure 1: The natESM ecosystem consists of core components, extended model components, technical infrastructure, and community services. Together, they form the foundation for open development, software that can be used over the long term, and the integration of new scientific requirements.
Open development requires more than just open-source code
Today, modern scientific software is often developed openly. The various source codes are publicly available (open source), version control systems enable collaborative work, and many model development projects rely on transparent development processes. However, our experience shows that while open-source code is an important prerequisite for collaborative development, it by no means guarantees open-minded development.
As a result, many scientific software projects face the challenge of ensuring model stability and scientific quality on the one hand, while also being able — and willing — to incorporate new ideas, contributions, and developers from the community on the other. This so-called “open development” requires more than just freely accessible code. It requires processes that enable the responsible model teams to preserve the scientific and technical integrity of their models, while at the same time allowing new developments to be transparently discussed, tested, and — if they prove themselves — integrated back into the main development branch. Open development requires more than open software. It requires the structures that allow communities to develop software together over the long term.
__________________________________________________________________________________
“In my experience, open development has the potential to break down silos. It’s not just about code; it’s about creating a framework where teams align their efforts and discover unexpected synergies.”
— Roland Potthast (German Weather Service, DWD)
__________________________________________________________________________________
Sprints as a bridge between scientific ideas and long-term usable software
Many developments in Earth system modeling begin with a scientific idea: a model needs to be expanded, a new component integrated, a numerical method improved, or the performance of existing code enhanced. However, the technical expertise to consistently implement such projects alongside day-to-day scientific work is often lacking.
To provide targeted support to model-developing institutions in this regard, natESM has established its own sprint process, through which scientists can work together with our Research Software Engineers (RSEs) to overcome technical challenges during one (or more) coding sprints lasting up to six months.
The process begins with what is known as a “sprint check.” Researchers contact natESM with their idea for improving their model code. During the sprint check, they discuss the initial technical situation, the task at hand, and possible solutions together with the RSEs.
This process also involves assessing whether a model meets the technical prerequisites for long-term integration into the natESM system. Modern Earth system models must be capable of running efficiently on current and future high-performance computers, support open development processes, and be technically maintainable. The close connection to the DKRZ facilitates direct exchange between model developers, RSEs, and experts in scientific high-performance computing. This allows the requirements of modern computer architectures to be incorporated into the technical development of the models at an early stage. Not all legacy software meets these requirements. The sprint checks therefore not only assist in planning developments but also in assessing whether an investment of community resources makes sense in the long term and will remain usable over the long term.
__________________________________________________________________________________
“The sprint check is like a first dance: we figure out if the rhythm fits.”
— Wilton Jaciel Loch (DKRZ)
__________________________________________________________________________________
Based on this technical analysis by the RSEs, applicants receive recommendations for a complete sprint proposal. This proposal is then reviewed by several parties: the entire natESM-RSE team, the natESM process coordinator, and the members of the natESM steering committee. The steering group consists of leading scientists from various disciplines within the German Earth System Modeling community. Our multi-stage sprint application process ensures that both long-term technical utility and scientific relevance are taken into account. This allows applicants to benefit from technical feedback and practical recommendations as early as the application phase.
If a sprint is approved, an intensive collaboration lasting up to six months begins between one of our RSEs and the participating scientists. Weekly meetings guide the technical implementation and help adapt the code step by step to modern computer architectures and future requirements. The sprint process is supplemented by two formal status meetings with the other natESM RSEs and the process coordinator. These meetings help identify potential difficulties early on, promote exchange and knowledge transfer among all participants, and, if necessary, jointly develop solutions for the remainder of the sprint.
The goal of a sprint is therefore not solely to produce functioning code. Equally important are transparent development processes, documentation, knowledge transfer, and the long-term usability of the developed solutions. We insist that the sprint results must be integrated back into the main development of the respective models so that they can be made available to the entire community.
Upon completion of the sprint, a joint report is drafted by the responsible RSE and subsequently finalized in collaboration with the researchers. The report undergoes a technical review by the steering committee. This additional scientific evaluation is a distinctive quality feature of the natESM sprint process. Few development projects benefit from such close guidance by proven experts from various disciplines.
The sprint reports are then published on the natESM website, and the results are presented by the participating researchers at the annual community workshop. In this way, experiences are not limited to individual project groups but become visible and usable for the entire community.
An idea originating in the scientific community thus becomes not just a technical development but a shared learning process from which the entire community can benefit in the long term.
How a sprint leads to shared infrastructure
One example is the integration of MESSy into the open extension interface ComIn from ICON. For many years, further developments of MESSy were closely tied to ICON’s internal structures. Every major change in the ICON code required extensive adjustments within MESSy, which made long-term maintenance difficult and slowed the adoption of new ICON versions.
Figure 2: From direct coupling to shared infrastructure: The integration of MESSy via ComIn reduces technical dependencies, facilitates future developments, and supports open development within natESM.
Another sprint example: HAMOCC on NEC Aurora
Together with scientists from the DWD, natESM-RSE Sergey Sukov (JSC) optimized key routines of the ocean biogeochemical model HAMOCC on the NEC Aurora Vector Engine. The vectorization of selected routines reduced runtime by up to a factor of 70 and increased the performance of coupled ICON-XPP simulations from 4.6 to 17 simulation years per day (SYPD). The optimizations were integrated into the ICON master and are thus available to the entire modeling community.
In two natESM sprints, RSEs and researchers worked together to gradually resolve these dependencies. The goal was to connect MESSy to ICON via the new ComIn interface, thereby creating a more sustainable and open development structure.
The result extends far beyond a single technical implementation. The direct coupling between the two systems was reduced, an initial complex ComIn plugin was developed, and a reference example for future extensions was created. At the same time, a common technical foundation was established that facilitates collaboration between two modeling groups that have evolved over decades.
The sprint exemplifies what natESM means by “open development”: not just open-source code, but jointly developed structures that enable long-term collaboration and sustainably support the further development of complex scientific software.
For the participating scientists as well, the added value often lies not solely in the technical results. Trang van Pham (German Weather Service, DWD) emphasizes that the work on performance optimization of ICON-XPP would hardly have been possible without the natESM sprint structure: “We simply do not have the resources for such a short technical project.”
Other sprint initiators also report similar experiences. For example, Clara Bayley from the Max Planck Institute for Meteorology describes the benefits of collaborating with natESM RSE Wilton Jaciel Loch as follows: “The progress Wilton made in such a short time would have taken me at least a year.”
More than software developement
However, the technical work is only one part of a successful sprint. Equally important is collaboration within the sprint team — between people with different backgrounds, experiences, and expectations. Researchers contribute deep subject matter expertise and have specific research questions in mind. The RSEs possess expertise in sustainable software engineering, performance optimization, and modern development processes. Both perspectives are necessary and must first come together during a sprint.
To ensure that different work cultures, perceptions of time, or expectations regarding roles and responsibilities do not lead to misunderstandings, natESM consciously invests in structures that promote exchange among all participants: weekly meetings between applicants and their assigned RSE are a must for every sprint; and the sprint checks and status meetings create spaces where expectations can be clarified, problems identified early on, and solutions developed collaboratively. This allows both sides to gain insights into each other’s working methods and challenges.
Ultimately, successful sprints are based on trust, respect, and a willingness to learn from one another.
__________________________________________________________________________________
“Project-based collaboration between dedicated research software engineers and scientists not only enables the efficient resolution of technical challenges but also facilitates the lateral exchange of knowledge within short project timelines.”
— Julia Nabel (Max Planck Institute for Biogeochemistry)
__________________________________________________________________________________
Spaces for exchange and collaboration
In addition to the sprints, natESM also supports professional exchange among community members. To this end, natESM offers various event formats that complement one another.
Our annual community workshops provide a space for discussing scientific and technical developments, new ideas, and shared challenges. In addition, any community member can propose topic-specific focus workshops to bring together experts from individual fields and discuss specific issues in depth.
One example of the outcome of such a focus workshop is the working group on ocean biogeochemistry, led by Judith Hauck (Alfred Wegener Institute for Polar and Marine Research) and Carsten Lemmen (Hereon), which was formed during the focus workshop initiated by Judith Hauck. The working group brings together researchers from various institutions to develop shared perspectives on the future development of biogeochemical modeling within natESM.
One of the working group’s initial outcomes is a sprint check proposal—an example of how the exchange among researchers from different institutions, all ultimately sharing the same goal — namely, the best possible representation of biogeochemical processes in the natESM ocean model — combined with the technical expertise of the natESM RSEs, can lead to concrete development activities for a new model for the “extended core components” of the natESM system. Initial surveys within the working group also show a high level of openness toward jointly developed model components supported by natESM.
In addition, we offer at least one technical training session each year, designed to impart practical skills related to the models, tools, and infrastructure components of the natESM system and, in particular, to help new community members get started with Earth system modeling.
All of our formats aim to create opportunities for exchange, discussion, and new collaborations. It is not uncommon for ideas for joint developments, new research projects, or additional sprint proposals to emerge here. In this way, knowledge is not only shared but also further developed collaboratively.
The next generation
Supporting the next generation of researchers has been one of natESM’s central goals from the very beginning. Against this backdrop, a natESM master’s course was offered for the first time during the 2025–26 winter semester at the University of Cologne, organized in close collaboration with Vera Schemann and with the participation of numerous researchers and research software engineers from the natESM community. The course not only covered technical content related to Earth system and climate modeling but also offered students direct insights into models, development processes, and research infrastructures within natESM (à course agenda).
With the first natESM Summer School in September 2026, this concept is now taking a step further. Early-career researchers from various institutions and universities will work there with a specially prepared natESM configuration that enables a safe and accessible introduction to Earth system and climate research. They will be accompanied by researchers from the natESM community, who will be available on-site as subject matter experts, as well as by natESM RSEs, who will provide technical support and teach the fundamentals of modern high-performance computing. The summer school is designed not only to impart technical skills but also to integrate early-career researchers into the natESM community at an early stage and familiarize them with the principles of open development.
The future depends on collaboration
Germany has been investing in scientific excellence, model development, and high-performance computing for decades. However, for these investments to continue to have their full impact in the future, more is needed than just hardware and software. It requires structures that connect scientific expertise, research software development, and modern computing infrastructures.
Together with partners from the German Earth System Modeling community, the natESM project — coordinated by the DKRZ — develops and operates precisely such structures. The goal is to ensure the long-term usability of knowledge, software, and technical expertise; to facilitate the adaptation of models to new computer architectures; and to strengthen collaboration across institutional boundaries.
__________________________________________________________________________________
“natESM is an excellent way to use limited resources in a focused manner, to build and share expertise, and to bridge the gaps between HPC, software engineering, and scientific rigor.”
— Patrick Jöckel (German Aerospace Center, DLR)
__________________________________________________________________________________
The future of Earth system modeling depends not only on the performance of new computers or the quality of individual models. It also depends on whether we succeed in bringing together people, models, software, and scientific expertise over the long term. To this end, the natESM project — coordinated by the DKRZ — is working with its partners to develop and operate a shared infrastructure for German Earth system modeling.
Further information:
- natESM website: www.nat-esm.de/
- Literature: natESM strategy paper as a pdf file
Author:
- Dr. Iris Ehlert, Process coordinator natESM, Department Application Support, DKRZ
natESM is funded by the Federal Ministry of Research, Technology, and Space (BMFTR) under Funding Agreement No. 01LK2107A1.