You are here: Home / Systems / Gitlab-runner

Gitlab-runner

This content will be added to gitlab.dkrz.de, once ready

Using gitlab CI

GitLab offers a continuous integration service. If you add a .gitlab-ci.yml file to the root directory of your repository, and configure your GitLab project to use a Runner, then each commit or push, triggers your CI pipeline. Detailed documentations on the topic can be found here.

The DKRZ welcomes and supports the use of gitlab CI to improve and ease the process of code maintenance. Yet, due to the fact, that the configuration, authorization and authentication of gitlab CI is possible only on gitlab-project level, not on user level, the use to test and run code directly on mistral is limited and currently not supported.

Yet, DKRZ offers runners to automatically build and publish the documentation of your code.

General notes on the usage of Gitab-CI

When using gitab CI for your project, you should be aware, that your CI can be controlled by the .gitlab-ci.yml file, located in the root directory of your repository. That implies that:

  • it can be viewed by anyone allowed to view your repository, so do not include any sensitive information. Use instead secret variables to pass sensitive informations to your job, if necessary.
  • potentially anybody allowed to commit changes, can alter your .gitlab-ci.yml and by this control your CI jobs. Thus, you need to carefully configure your CI (by means of your .gitlab-ci.yml and your gitlab-project management interface), so that it is triggered and changed only if and by whom you want it to. The DKRZ runners are protected by default, meaning they can only be triggered by protected branches or protected tags. We further recommend to use the only/except keyword in your .gitlab-ci.yml to explicitly define what triggers your job (e.g. certain branches, tags etc.). It is also possible to manually exclude a certain commit from the CI by adding [ci skip] or [skip ci] to the commit message.

Building your documentation and publishing it with gitlab pages

Currently, the DKRZ offers two specific runners with a docker executor (with a basic alpine image) to automatically build your documentation and publish and host it with gitlab pages along with you repository.

Note: Your documentaion published and hosted on gitlab pages is open to the public. Currently, there is no way to limit access to only authorized users.

 

The .gitlab-ci.yml:

To enable and configure gitlab CI you need to add a .gitlab-ci.yml file in the root directory of your repository. We offer templates for doxygen and sphinx, that cover most standard installations:

To adapt the corresponding template to your needs, please read the details on the configuration below.

Validating the .gitlab-ci.yml:

To validate your GitLab CI configurations, you can use CI lint, located at https://gitlab.dkrz.de/*your-project*/-/ci/lint (you also find the link to CI Lint at "CI/CD")

Triggering and monitoring your job:

By commiting changes to a protected and authorized branch, your job will be triggered. You can monitor the status, progress and results of your jobs under "CI/CD". Here you can also check the console output of your jobs or restart a failed job.

Your docu page:

After your job has finished successfully you will find the website with your documentation under http://namespace.gitlab-pages.dkrz.de/your-project, where your project is the name of your project and namespace is defined by your username on the gitalb instance, or the group name you created your project under. You can also look up the url of your page in the "pages" section of your project settings.

Configuration of your .gitlab-ci.yml in detail:

We explain here the details of the offered template file, so you can adapt it to your needs. Although we use here the template file to build doxygen pages, the explanations are applicable for all templates.

For further details on the configurations options of your job, please refer to  the official gitlab documentation.

pages:

 script:
 - doxygen doc/Doxyfile
 - mv doc/html/ public/

 tags:
 - doxygen

 only:
 - master

 artifacts:
  paths:
  - public
  expire_in: 5min

pages specifies the name of the job and defines the beginning of the job's configuration

script contains the commands, executed on the docker image. In this template the doxygen pages are build and then moved to the public directory, from which they will be automatically published on gitlab pages. Any further commands needed for your product to build the documentaion need to be added here.

Note that the DKRZ runners use an alpine linux with pre-installed doxygen, graphviz, ghostscript and latex (texlive) for the doxygen runner and pip, sphinx and latex (texlive) for the sphinx runner. Any further packages needed by your application to build the pages first need to be installed via apk add.

 

tags identifies the runner to be used, by selecting only runners with the set tags. The offered DKRZ runners only run tagged jobs, thus in this example the "doxygen" tag is needed to use the runner.

only defines the branches (or tags) that are triggering the job. It should bes set to avoid unauthorized as well as unintended triggering.

Note that there is only one website per project. That implies that an unauthorized or unintended triggering of the pages job will override the projects documentation with the version produced by the commited branch. Thus you should always set the only keyword to determine which branches are allowed to trigger the job.

 

artifacts contains all information concering the files, that should be transfered from the docker image to the gitlab instance. All other files - except for the console output - are removed with the image. The produced documentation (moved to the public directory before) need to be copied to the gitlab instance and are referenced by the paths keyword. expire_in defines the time after which the transfered files (artifacts) are removed. Since all files in the public directory on the gitlab instance will be automatically copied and deployed on gitlab-pages, the transfered files (artifacts) can and should be removed after some few minutes, to make sure that the diskspace limit of your project is not exceeded.

Note that your documentation might be rather large. To save diskspace always set the expire_in keyword to a few minutes, so that your files will be removed after they have been deployed.

 

Document Actions