Skip to content

Deploy Jenkins integration to production GitLab

Software Heritage migration from Phabricator to GitLab should be effective on Monday 2023-01-09.

During the last weeks, I took care of modifying our Jenkins jobs configuration to add GitLab integration. I used our GitLab staging instance to test that integration.

In order to not break the Phabricator jobs until the migration to GitLab effective, I created new jobs from the existing ones dedicated to be triggered by GitLab webhooks.

The jobs modifications for GitLab mainly consisted in changing git clone URLs and using the features offered by the Jenkins GitLab plugin, notably setting build triggers and reporting the build statuses on commits and tags.

This task summarizes what must be done to effectively deploy Jenkins integration to our production GitLab once it is ready to use:

  • Switch to production GitLab URLs and settings in Jenkins jobs configuration (swh/infra/ci-cd/swh-jenkins-jobs!124 (merged))
  • Create a Jenkins user in our production GitLab and :
    • Generate a GitLab token with api scope and no expiration date for it and save it in our credentials repository under //operations/gitlab-production/jenkins/api-token
    • Generate a pair of public/private SSH keys for it and save them in our credentials repository under //operations/gitlab-production/jenkins/ssh-keys
    • Associate the public SSH key to the Jenkins user account on GitLab (required for jobs building debian packages as they need to push tags on git repositories)
  • Modify Jenkins settings to allow proper communication with our production GitLab:
    • Create or update the following credentials:
      • Kind: GitLab API token, API token: <the GitLab API token for Jenkins user>, ID: gitlab-token
      • Kind: Secret text, Secret: <the GitLab API token for Jenkins user>, ID: jenkins-gitlab-token
      • Kind: SSH Username with private key, Username: jenkins, Private key: <copy/paste private key of Jenkins user on GitLab>, ID: gitlab-public-ci-ssh
    • Create a new GitLab connection for Jenkins through the Gitlab section of Jenkins Configure System page:
      • Ensure Enable authentication for '/project' end-point option is unchecked (triggering jobs is already protected by a secret token so we can disable it to simplify GitLab integration)
      • Create new connection: Connection name: gitlab-swh, Gitlab host URL: https://gitlab.softwareheritage.org, Credentials: <select "GitLab API token" in dropdown>
  • Setup Jenkins integration on our production GitLab by executing the jenkins-tools/swh-jenkins-jobs-builder job that will update jobs configuration and setup GitLab webhooks to trigger jobs
  • Remove Phabricator related jobs in Jenkins jobs configuration once migration to GitLab effective (swh/infra/ci-cd/swh-jenkins-jobs!149 (merged))

Once Jenkins integration on GitLab effective, its configuration will be automatically updated when pushing commits to the master branch of the swh-jenkins-jobs repository hosted on GitLab (successful Jenkins configuration files generation will also be checked when submitting merge requests to that repository).

Migrated from T4748 (view on Phabricator)

Edited by Antoine Lambert
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information