You are here: Home / Systems / gitlab.dkrz.de

gitlab.dkrz.de

DKRZ hosts a web-based Git repository manager based on the GitLab Community Edition.

Prerequisites

GitLab access is available for all DKRZ users.

In order to use the GitLab resource, project administrators need to activate the "Gitlab" permission per user on luv.dkrz.de before a user can login to gitlab.dkrz.de with his/her LDAP password.

Once activated, each user is allowed to create up to 5 git projects/repositories - please contact Beratung
if you need more resources.

Please note that GitLab was mainly developed for the versioning of src code. Therefore, we kindly ask you to not store large files or binary code and to keep the size of each repository below 100MB. Currently, there is no explicit quota set, but we will inform you whenever usage exceeds the threshold of 100 MB. In seldom cases we will have to temporarily deactivate your access if a misuse of the service is detected. If you need to store large files within Git, we recommend to take a look at the Git Large File extension where DKRZ provides a binding to our cloud storage.

Usage

You can access the DKRZ GitLab website at https://gitlab.dkrz.de

New projects

As a new user you will want to add an SSH key to your GitLab profile. This allows you to access project repositories with a Git client without having to type your password every time. You can add an ssh key to your GitLab account at https://gitlab.dkrz.de/profile/keys.

Projects can have different visibility settings:

  • Public - visible to everyone
  • Internal - visible to other DKRZ GitLab users
  • Private - visible only to project members

When you add new members to your project, you will assign them to a role. You can read about project permissions at https://gitlab.dkrz.de/help/user/permissions.

Existing projects

Cloning an existing repository using SSH keys is done via:

$ git clone [Email protection active, please enable JavaScript.]:me/my_project.git

You have to upload your public SSH key to the gitlab server first! Alternatively, one can use HTTPS protocol to clone a repository:

$ git clone https://gitlab.dkrz.de/me/my_project.git

On mistral you should use the latest Git version when working with GitLab:

$ module add git

If you are not familiar with GitLab, you might have a look at the official documentation:

http://doc.gitlab.com/ce/

There is also a small tutorial available: GitLab DKRZ tutorial

To get started with Git repositories and version control, we cross-refer to these sites:

 

Using Git for large files

Managing large files such as audio, video and graphics files has always been one of the shortcomings of Git. The general recommendation is to not have Git repositories larger than 1GB to preserve performance. At DKRZ we kindly ask our users to not exceed 100MB per Git repository.

Nevertheless, large files can be managed by Git using the Git LFS client (https://git-lfs.github.com/). It replaces large files with text pointers inside Git, while storing the file contents on a remote server. DKRZ provides the option to store these files on the cloud storage (https://www.dkrz.de/up/systems/cloud).

As a prerequisite to use Git LFS user have to

  • install Git LFS extension - all git modules on mistral already have the extensions installed
  • prepare an empty container at DKRZ cloud storage, e.g. via Swiftbrowser accessible at https://swiftbrowser.dkrz.de

 

Working with Git LFS requires only some small changes:

On mistral: make sure that you are using one of the git modules git/2.4.6 , git/2.5.0, git/2.9.0 or git/2.11.2

 

Clone the repository using explicit lfs command - if the repository is already using Git LFS:

$ git lfs clone [Email protection active, please enable JavaScript.]:user/repo.git

Depending on your git lfs version, there might be a message

batch response: Git LFS is not enabled on this GitLab server, contact your admin.

you can safely ignore this - all data (except those from the LFS server) are cloned already.

 

For new local repositories that use Git LFS (and also for bare repositories that have not been setup for Git LFS), one needs to specify the LFS server – in the example 'container' should be the name of the swift container that was prepared initially

$ cd repo
$ URL="https://git-lfs-swift.dkrz.de/container"
$ git config lfs.url $URL
$ git config lfs.$URL/.access basic
$ git config lfs.batch true
$ git config --file=.lfsconfig lfs.url $URL

Next steps are either A or B1+B2

A/ If the mandatory .gitattributes and .lfsconfig files are already present in the repository, one can directly pull all data from the LFS server

$ git lfs pull

B1/ If no Git LFS was used for this repository so far, you have to specify which files are to be stored on the cloud storage – the example below uses all png files

$ git lfs track "*.png"

B2/ Add .gitattributes and .lfsconfig file to repository to mark it as LFS repository

$ git add .gitattributes .lfsconfig

Now use the repository as usual, e.g. add and commit

$ git commit -m 'git lfs setting added'

When pushing or pulling data to/from Git LFS repositories, one has to use the modified git lfs commands, e.g.

$ git lfs push origin master

This will contact the git-lfs server and ask for swift credentials authentication where

  • Username is 'swift-account;ldap-username'
  • Password is your ldap password

All tracked files (in this example all png files) will now be stored at cloud storage in the specified container for specified swift-account.

Document Actions