26.02.2015

On July 1., the first phase of the new High Performance Computer System for Earth System Research (HLRE-3), Mistral, has started its regular operation. It consists of computer components by Bull, a disk storage system by Xyratex/Seagate and high performance network switches by Mellanox. These components are distributed over 41 racks weighing up to or even more than a ton, which are connected by bundles of fiber fabric.

About 1,500 compute nodes on the basis of Bullx B700 DLC systems with two 12-core Haswell processors each (a total of 36,000 cores) will reach a peak performance of 1.4 PetaFLOPS. This would be an increase by a factor of 9 compared with the Blizzard. On the list of the fastest computers of the world, the Top500 list, Mistral was rankend 56th in July 2015. A remarkable innovative feature of the new system is the warm water cooling system, a technology that requires significantly less energy than standard cooling systems, as the heat generated by processors and memory modules is extracted as close as possible to source.

HLRE-3 Phase 1 setup

 

At 120 Terabyte, the system's main memory is 6 times larger than that of the IBM, and the new Lustre-based parallel file system will offer 20 Petabyte, being more than three times larger than the current disk system. Furthermore, 12 visualization nodes equipped with NVidia Tesla K80 GPUs and 50 nodes for interactive access and pre- and post-processing of simulation data complement the system.

The second phase of HLRE-3 is planned to be available in summer 2016. This extension will additionally roughly double computing and disk storage capacity. With a peak performance of 3 PetaFLOPS and a 50 Petabyte parallel file system, scientists can improve the regional resolution, account for more processes in the Earth system models or reduce uncertainties in climate projections.

HLRE-3 Cabling

The entire HLRE-3 project with a total funding of 41 million euros also comprises diverse modifications and extensions of DKRZ's electrical and cooling infrastructure, a substantial extension of the data archive system as well as a research cooperation with Bull - aimed at the development of more efficient computing algorithms for climate modeling. Two thirds of the funding is supplied by the German Federal Ministry for Research and Education (BMBF). The remaining part is financed by the Helmholtz Association (Helmholtz-Gemeinschaft Deutscher Forschungszentren). The Helmholtz-Zentrum Geesthacht - Centre for Materials and Coastal Research (HZG), one of DKRZ's shareholders, is responsibe for organizing the proper flow of funds.

HLRE-3_Storage@work

 

Parts of the HLRE-3 system such as the storage racks shown left are already powered on in order to configure and test the installation.

 

The information on the transition from HLRE-II to HLRE-3 in our user portal will be updated and detailed on a regular basis.

Presse releases:

  1. Press release from 27th February 2015 about the start of the installation of HLRE-3 (in German) as a pdf file for Download.
  2. Press release from 8th May 2015 about the new upgrade of the archiving system of HLRE-3  as a pdf file for Download.
  3. Press release from 13th July 2015 about Mistral starting its operation as a pdf file for Download.

 

Mistral Nodes running

Save-the-Date of the official inauguration:
DKRZ will inaugurate its new high performance computing system „Mistral“ on 5th October 2015 and invites all its users, partners and the DKRZ team.
In the afternoon and in the morning of 6th October a scientific workshop will give DKRZ users the opportunity to present their latest projects and to exchange their first experiences with the new system. In the afternoon of the 6th October and in the morning of 7th October DKRZ will give a training course on the best practices of using the new system.
Save-the-Date-invitation (in German) as a pdf file for download.