Sie sind hier: Startseite / Systems / jupyterhub.dkrz.de
Info
Alle Inhalte des Nutzerportal sind nur auf Englisch verfügbar.

jupyterhub.dkrz.de

Jupyter offers the possibility to interactively work with Python in a web browser. Jupyterhub enables DKRZ users to run the workload directly on the Mistral HPC system - with access to all their data in the lustre filesystem.
DKRZ operates the jupyterhub server in extended testing phase, which means that we do not guarantee fulltime availabilty of the system and some more limitations documented here.

What are Jupyter Notebooks?

So called "notebooks" are documents that can contain Python (or other) code and further text elements like comments, figures, equations, etc. They are human-readable and executable by Jupyter in order to produce the desired results.

What is Jupyter?

Jupyter (formerly known as IPython) itself is a web application that allows editing and running notebooks via a web browser. Each notebook uses its own "kernel" that determines the used Python (or other language) version and associated modules.

What is JupyterHub?

JupyterHub is a multi-user server for Jupyter notebooks that allows to execute the notebook directly on the DKRZ HPC system Mistral. It therefore also supports the execution of parallel computation.

JupyterHub is available at https://jupyterhub.dkrz.de for all DKRZ users who have access to Mistral and who are allowed to submit batch jobs.

Using jupyterhub.dkrz.de

Navigate to the https://jupyterhub.dkrz.de URL using a web browser and log in with your DKRZ account.

After login to the jupyter web interface, press the 'Start My Server' button which opens the options menu for the notebook server to be spawned.

Several job profiles are predefined. Choose the profile that best fits your workload:

profile good for ...
shared partition; 1,2 or 4 cores (2.5 GByte memory per core) small development/testing projects
prepost partition; 1,2 or 4 cores (2.5 GByte memory per core) as before, but allowing access outside of mistral (wget or similar)
compute partition: 24 cores, 64 GByte memory heavy workload that needs lots of memory and/or parallel execution

You also need to specify the project account such that used the CPUtime gets accounted correctly. The "Reservation" option just needs to be filled, if you wanna use a specific node reservation within SLURM, e.g. for workshops.

By clicking on 'Spawn' a new notebook server is submitted as a batch job on the HPC system Mistral. You will be redirected automatically once the SLURM allocation is ready to start the server.

If the batch queue is filled with waiting jobs, it might take some time to start the notebook server. There is an internal timelimit specified until which the resources should be ready. If not the spawner will reject and ask you to start again (maybe with reduced resources specified).

Once your session has started you should see the Jupyter file browser interface.

Use the New menu within the Files tab to start a new Notebook or Terminal. It will be opened in a new browser tab.

Managing Jupyter sessions

To see what Jupyter Notebooks and Terminals your Jupyter server is running, use the home browser tab and click on the Running tab. Here you can shutdown resources if needed.

To stop the entire Jupyter Notebook server itself, use the 'Control Panel' in the upper right part of the dashboard and click on 'Stop My Server'.

There are a few points you should be aware of when working with Jupyter Notebooks on the jupyterhub server
  • Sessions are persistent. If you close the browser tab without stopping the server before, you can revisit the jupyterhub site and carry on working
  • SLURM might cancel your server session if a timeout occurs after 8 hours
  • stdout and stderr of the SLURM batch job that is spawner by jupyterhub is written to your HOME directory on the HPC system in file "jupyterhub_slurmspawner_<id>.log"

More documentation needed?

For all details refer to the official jupyter documentation.

A collection of example notebooks on how to use this tool for postprocessing climate data is available at the DKRZ Gitlab server: jupyterhub_notebooks

You might also have a look at the rendered version of the notebook 'Jupyterhub_features.ipynb' to immediately see the possibilities: Jupyterhub_features.ipynb

Artikelaktionen