Configure a self‑hosted GitLab CI runner with Docker and remote cache: practical tutorial

Deploy & DevOps · By · Published on · Updated on · 8 min read

Colorful shipping containers stacked under a blue sky
Visual metaphor for Docker containers and image registries Photo: Jan van der Wolf / Pexels

This tutorial walks you step by step to set up a self-hosted GitLab CI runner in Docker, register it, enable S3/MinIO remote cache and use BuildKit with registry cache to speed up builds. It includes an example .gitlab-ci.yml and commands to verify everything is working.

Architecture and cache choices

You'll run the GitLab Runner inside a container using the Docker executor, which isolates jobs in containers. We'll use two cache mechanisms to cut pipeline time and network traffic: the runner file cache and image layer cache via BuildKit in the registry. Together they cover application dependencies and Dockerfile layers.

  • Runner Cache (S3/MinIO): stores dependencies and temporary directories (node_modules, vendor, .m2) between jobs.
  • Docker image layer cache (BuildKit → registry): reuses image layers across builds in the Container Registry.
Feature Runner Cache (S3/MinIO) Docker registry cache (BuildKit)
What it stores Dependencies and temporary directories Dockerfile image layers
Scope Per job/project; can be shared Per Dockerfile/tag; shared across refs
When it acts Before/after the job (restore/save) During docker buildx build
Required infra S3-compatible bucket (e.g., MinIO) Registry (e.g., GitLab Container Registry)
Typical benefit Speeds package managers and app builds Speeds image rebuilds and layer push/pull

Prerequisites

Make sure you have the items below before starting. You will need administrative privileges on the runner host.

  • An accessible GitLab instance and a runner registration token.
  • A Linux server with root access and outbound internet connectivity.
  • Docker installed on the host. If you need installation steps, see: How to Install Docker and Docker Compose on Linux.
  • Optional for runner cache: an S3/MinIO bucket accessible from the server.
  • Access to the project's GitLab Container Registry for BuildKit layer caching.

Start and register the GitLab Runner (Docker executor)

We'll create the config volume, run the Runner container and register it. The executor will be Docker, with access to the host socket so jobs can create containers. Keep the registration token secure. The non-interactive register command below should be adjusted for your GitLab URL, token, runner name and versions.

# 1) Preparar diretório de configuração
sudo mkdir -p /srv/gitlab-runner/config

# 2) Subir o container do GitLab Runner
sudo docker run -d --name gitlab-runner \
  --restart always \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:alpine

# 3) Registrar o runner (ajuste URL e token)
sudo docker run --rm -it \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  -v /var/run/docker.sock:/var/run/docker.sock \
  gitlab/gitlab-runner:alpine register \
  --non-interactive \
  --url "https://gitlab.example.com/" \
  --registration-token "REG_TOKEN" \
  --name "runner-docker" \
  --executor "docker" \
  --docker-image "docker:24.0" \
  --docker-privileged \
  --docker-volumes "/cache"

Adjust config.toml for Docker and BuildKit

After registration, edit config.toml to enable BuildKit and set volumes. BuildKit speeds up builds and improves layer caching. We keep the executor privileged for scenarios that require DinD. To optimize your image layers, see guidance on creating lightweight images and cache strategies in the linked resources.

concurrent = 2
check_interval = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "runner-docker"
  url = "https://gitlab.example.com/"
  token = "TOKEN_GERADO_NO_REGISTRO"
  executor = "docker"
  environment = [
    "DOCKER_DRIVER=overlay2",
    "DOCKER_TLS_CERTDIR=",
    "DOCKER_BUILDKIT=1",
    "BUILDKIT_PROGRESS=plain"
  ]
  [runners.docker]
    image = "docker:24.0"
    privileged = true
    tls_verify = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]

Enable S3/MinIO remote cache for the Runner

Now configure the remote cache to restore and save dependencies faster. If you use MinIO without TLS, enable Insecure. Restrict bucket access to a dedicated user. Place the block below inside your [[runners]] section in config.toml and adjust endpoint, region, bucket and credentials for your provider.

  [runners.cache]
    Type = "s3"
    Shared = true
  [runners.cache.s3]
    ServerAddress   = "minio.internal:9000"
    BucketName      = "gitlab-cache"
    BucketLocation  = "us-east-1"
    AccessKey       = "MINIO_ACCESS_KEY"
    SecretKey       = "MINIO_SECRET_KEY"
    Insecure        = true

Pipeline (.gitlab-ci.yml) with BuildKit and registry cache

This example builds and publishes the image to the GitLab Container Registry using BuildKit with cache stored in the registry. That reuses layers between builds, reducing time for repeated Dockerfile steps. The docker:dind service exposes the daemon on 2375 without TLS. Use this only in controlled environments. Registry credentials come from standard GitLab CI variables.

image: docker:24.0

services:
  - name: docker:24.0-dind
    command: ["--registry-mirror=", "--mtu=1460"]

variables:
  DOCKER_HOST: tcp://docker:2375
  DOCKER_TLS_CERTDIR: ""
  DOCKER_DRIVER: overlay2
  DOCKER_BUILDKIT: "1"
  BUILDKIT_PROGRESS: plain
  IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  CACHE_REF: $CI_REGISTRY_IMAGE:buildcache

stages: [build]

build-image:
  stage: build
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - docker buildx create --use --name builder || docker buildx use builder
  script:
    - docker buildx build \
        --push -t "$IMAGE" \
        --cache-from type=registry,ref="$CACHE_REF" \
        --cache-to type=registry,ref="$CACHE_REF",mode=max \
        -f Dockerfile .
  rules:
    - if: $CI_COMMIT_BRANCH

Verification and common troubleshooting

Run the commands below to validate logs and connectivity. The first build may take longer; subsequent builds should fetch fewer dependencies and skip Dockerfile steps that were cached.

  • If the runner doesn't show up in GitLab, check the token and URL.
  • If the S3 cache isn't working, validate endpoint, keys and bucket policy.
  • If buildx fails, check Docker and dind service versions.
# Ver logs do runner
sudo docker logs --tail=200 gitlab-runner

# Verificar conectividade e estado do runner
sudo docker exec -it gitlab-runner gitlab-runner verify --delete

# Conferir se o bloco de cache está ativo no config
echo "--- cache config ---" && \
  sudo docker exec -it gitlab-runner \
  grep -A4 "\[runners.cache\]" /etc/gitlab-runner/config.toml

Operation, security and maintenance

Keep the runner up to date and sized to your infrastructure. Aggressive cleanups can remove useful layers and caches — schedule them carefully. For host sizing and monitoring guidance, see the linked materials.

  • Update the runner container periodically and redeploy using the same config volume.
  • Limit concurrency (concurrent) to what the host can handle. Sizing guidance: How to choose the right VPS size.
  • Monitor CPU, RAM and disk. Instrument with Prometheus & Grafana via Docker Compose.
  • Be careful with prune: it can remove images and caches; run it outside peak hours.
# Atualizar imagem do runner e inspecionar estado
sudo docker pull gitlab/gitlab-runner:alpine
sudo docker exec -it gitlab-runner gitlab-runner list
sudo docker exec -it gitlab-runner gitlab-runner status

# Limpeza com cautela (pode remover camadas e caches)
sudo docker system prune -af --volumes

Frequently asked questions

Do I need Docker-in-Docker or the host socket?

For the Docker executor it's common to mount the host socket into the runner container. This avoids DinD overhead and is usually simpler. Use DinD when you want a fully isolated Docker daemon per job, accepting higher resource usage.

What's the difference between Runner cache and Docker (BuildKit) cache?

The Runner cache stores job files like language dependencies. The Docker cache stores image layers created during builds. They complement each other: the first speeds application tasks, the second speeds Dockerfile rebuilds.

Where are Runner cache files stored?

With S3/MinIO configured, cache artifacts are uploaded to the specified bucket. Without S3, cache stays local on the runner host, which makes sharing harder and risks loss when the container is recreated.

How do I limit concurrent jobs?

Set the concurrent key in config.toml to define the maximum simultaneous jobs per runner service. Balance it with server capacity. Monitor CPU, memory and I/O to find the right level without causing contention.

Can I share the same cache across different projects?

Yes, if you set Shared = true in the runner cache block and use compatible cache keys in jobs. Avoid generic keys to prevent data leaks. Separate buckets or key prefixes help isolate projects.

How to rotate S3/MinIO credentials securely?

Create a dedicated user with minimal permissions for the bucket and rotate keys regularly. Never commit credentials. Use environment variables in the runner container or your platform's secret management where possible.

Want to speed up your CI securely?

If you need help designing a self-hosted runner, configuring remote cache and optimizing Dockerfiles and pipelines, we can discuss your setup and provide an assessment.

Talk about your CI/CD

See also