Debugging jobs in GitLab CI

GitLab does not support SSH access to a job in order to debug it like other CI tools like CircleCI or Travis CI. Instead, it provides developers with the ability to set up a GitLab Runner to run jobs locally. This article explains how to run a job locally and halt it so you can jump into the container and debug it.

Background: a failing job at GitLab CI

When I wrote An Installer for Drupal 8 and GitLab CI, I created a demo repository to host a Drupal 8 site and wrote a GitLab CI Pipeline. This pipeline, among other things, ran end to end tests against the Drupal 8 site. These tests needed a browser so I added Headless Chrome, a popular browser to use in Continuous Integration which does not need a display to run.

As I was setting up the Existing Site Tests job, I saw that the job would get stuck when running the Chrome browser in headless mode. Like this:

If you look at the above console output, the job gets stuck there and eventually times out. The google-chrome-unstable  command is called by a Robo task. This was the job setup:


I needed to be able to stop the job right before vendor/bin/robo run:chrome-headless and somehow jump into the container so I could run that command manually to see what was going on. By looking at the GitLab documentation I found out that I needed to set up a runner.

Setting up a Docker runner

Searching for how to retry a failed job with SSH access I discovered that in GitLab this is not supported. Instead, the recommended approach is to install gitlab-runner and then register a Docker runner so you can run a job locally.

I started by installing the gitlab-runner command by following the steps listed at Install GitLab Runner manually on GNU/Linux:

$ curl -LJO https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb
$ sudo dpkg -i gitlab-runner_amd64.deb 

Next, I followed the instructions at https://docs.gitlab.com/runner/register and registered a runner, which asks a few questions along the way in order to configure it:

$ sudo gitlab-runner register
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
https://gitlab.com
Please enter the gitlab-ci token for this runner:
[Find this token at your repository's Settings > CI / CD > Runners ]
Please enter the gitlab-ci description for this runner:
debugging
Please enter the gitlab-ci tags for this runner (comma separated):
debugging
Registering runner... succeeded                     runner=NZu1SRU5
Please enter the executor: virtualbox, docker, docker-ssh, shell, ssh, docker+machine, docker-ssh+machine, kubernetes, custom, parallels:
docker
Please enter the default Docker image (e.g. ruby:2.6):
juampynr/drupal8ci:latest
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 

 Setup complete. Here is a screenshot of where to find the token to use above. I am posting it as it is not easy to find within the repository settings:
 
Now I could run jobs locally. In the next section we will test the runner.

Adjusting jobs so they can run locally

 Here is the command to run the job that I wanted to debug:

$ gitlab-runner exec docker drupal8ci:existing_site_tests