You are here: Home / News & Events / News / New Archival System at DKRZ: Time schedule and further information

New Archival System at DKRZ: Time schedule and further information

Jan 28, 2021

Dear users,

With this email we would like to provide some detailed information regarding the upcoming change (April 2021) in the way users archive and retrieve data from the DKRZ’s tape archive. This change affects ALL DKRZ users actively using DKRZ’s archiving infrastructure.

As announced in November 2020, a new Hierarchical Storage Management (HSM) system will be put in operation at DKRZ in April 2021. The HSM system called StrongLinkforms a new technical solution for the DKRZ tape archive system and replaces the current storage management system HPSS system. This is done in preparation for the installation of the new HLRE-4-cluster Levante and goes along with a replacement of the pftp command line tool used so far.

Today we would like to announce more details and a preliminary time schedule:

  • 28 Jan: release of a FAQ page regarding the new HSM system
  • 15 Feb (approximately): release of first version of the user documentation
  • Mar/Apr, tba: DKRZ Tech Talk on the new HSM system; will take place one or two weeks before the HPSS is taken offline (see next item)
  • Mar/Apr, tba: HPSS will be taken offline and data from HPSS will be migrated to the new system. See below for more information on which data will be migrated. This will probably take one week in March/April 2021 but not before 22 of March. Access to the HSM and archived data is not possible during the migration.  

The exact date for the migration and the DKRZ Tech Talk will be announced at least one month in advance of the migration. Before we fix these dates we have to make sure that the migration will run flawlessly.

All data, except those in the directory /dxul and subdirectories thereof, will be migrated to the new system. Data in /dxul are no longer accessible after the migration to the new system (see separate email on this topic).

A new command line tool "slk" (short for StrongLinK) for data archival, retrieval and search will replace the current tool pftp. pftp cannot be used anymore with the new system. The new tool will have very similar features to pftp and is suited for use in non-interactive workflow setups. It will be presented in the DKRZ Tech Talk. Please be aware that, depending on the complexity of your workflows, you will need to invest time in adapting your scripts to the new environment.

The StrongLink product offers a number of new features. As one of these, it will allow to automatically harvest and store extended file metadata of some file types. Files can be searched for by these metadata.

If you have specific needs for the archival of data in early April and need to test specific automatic archival workflows in advance, please contact us at [Email protection active, please enable JavaScript.].

You are welcome to contact us if you have any questions. Please have a look into the FAQ first to see whether your questions are already covered there. The FAQ is versioned so that you can easily find out whether there are updates or not. The current version is 1.01.

FAQ: https://www.dkrz.de/s/hsm_faq

Best regards,
DKRZ

Document Actions

Filed under: