17.11.2020

The key component of the new archive system is the powerful metadata-driven workflow engine called StrongLink from StrongBox Data Solutions (SBDS). The system designed by Cristie Data relies on StrongLink in connection with servers and particularly powerful OceanStor storage systems from Huawei, which act as temporary storage for the data to be archived.

Using the StrongLink architecture, the 150 Petabyte of climate model data of the last 30 years will be migrated automatically from the former High Performance Storage System (HPSS) into the new HSM within a week.

The new system manages the 9 existing tape libraries with a total of 79,000 cartridge slots and 90 LTO tape drives. The 1 Petabyte hard disk cache of the new HSM achieves a transfer rate of 15 Gigabytes/s. By using the open standard LTFS (Linear Tape File System) for data storage on the magnetic tapes, the climate data will be archived in a future-proof manner and the data exchange with other institutions can be simplified. In addition, climate data will be archived with supplementary metadata so that it can be retrieved more easily. The S3 interface contained in StrongLink also significantly expands the possibilities for providing data access at application level.

From mid-March 2021, when the new HSM will start its operation, up to 120 petabytes of newly generated data can be archived annually. Thanks to the scalability of the chosen solution, the total capacity can be adapted to further increasing requirements over the coming years, so that a data volume of up to 500 petabytes is expected by 2026. In principle, the storage concept allows data volumes of more than one exabyte (1,000 petabytes) to be archived and managed.

“This new system will enable DKRZ to expand and modernize essential data services to our research community, while also seamlessly transitioning to a new open-standard platform that can grow as we do.”, said Prof. Thomas Ludwig, CEO at DKRZ. “This will provide a solid foundation to ensure researchers have ongoing access to the critical data needed for climate modeling and improved climate projections.”

The installation, configuration and project management for the new HSM system at DKRZ will be provided by Cristie Data. Data Cristie will also provide all of the hardware needed for the project and will be responsible for the integration service and management.

„DKRZ is an important partner for climate research. At Cristie Data, we take our responsibility to the environment very seriously. Using StrongLink, climate researchers all over the world will be guaranteed to access their data fast and reliably in the future. With Huawei as one of the leaders for energy-efficient data center solutions, we are also making a contribution to more environmentally friendly solutions in the data center,” said Volker Wester, Managing Director of Cristie Data.