Configure a self‑hosted GitLab CI runner with Docker and remote cache: practical tutorial
Deploy & DevOps · By Kevin Marcondes · Published on · Updated on · 8 min read
This tutorial walks you step by step to set up a self-hosted GitLab CI runner in Docker, register it, enable S3/MinIO remote cache and use BuildKit with registry cache to speed up builds. It includes an example .gitlab-ci.yml and commands to verify everything is working.
Architecture and cache choices
You'll run the GitLab Runner inside a container using the Docker executor, which isolates jobs in containers. We'll use two cache mechanisms to cut pipeline time and network traffic: the runner file cache and image layer cache via BuildKit in the registry. Together they cover application dependencies and Dockerfile layers.
- Runner Cache (S3/MinIO): stores dependencies and temporary directories (node_modules, vendor, .m2) between jobs.
- Docker image layer cache (BuildKit → registry): reuses image layers across builds in the Container Registry.
| Feature | Runner Cache (S3/MinIO) | Docker registry cache (BuildKit) |
|---|---|---|
| What it stores | Dependencies and temporary directories | Dockerfile image layers |
| Scope | Per job/project; can be shared | Per Dockerfile/tag; shared across refs |
| When it acts | Before/after the job (restore/save) | During docker buildx build |
| Required infra | S3-compatible bucket (e.g., MinIO) | Registry (e.g., GitLab Container Registry) |
| Typical benefit | Speeds package managers and app builds | Speeds image rebuilds and layer push/pull |
Prerequisites
Make sure you have the items below before starting. You will need administrative privileges on the runner host.
- An accessible GitLab instance and a runner registration token.
- A Linux server with root access and outbound internet connectivity.
- Docker installed on the host. If you need installation steps, see: How to Install Docker and Docker Compose on Linux.
- Optional for runner cache: an S3/MinIO bucket accessible from the server.
- Access to the project's GitLab Container Registry for BuildKit layer caching.
Start and register the GitLab Runner (Docker executor)
We'll create the config volume, run the Runner container and register it. The executor will be Docker, with access to the host socket so jobs can create containers. Keep the registration token secure. The non-interactive register command below should be adjusted for your GitLab URL, token, runner name and versions.
# 1) Preparar diretório de configuração
sudo mkdir -p /srv/gitlab-runner/config
# 2) Subir o container do GitLab Runner
sudo docker run -d --name gitlab-runner \
--restart always \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:alpine
# 3) Registrar o runner (ajuste URL e token)
sudo docker run --rm -it \
-v /srv/gitlab-runner/config:/etc/gitlab-runner \
-v /var/run/docker.sock:/var/run/docker.sock \
gitlab/gitlab-runner:alpine register \
--non-interactive \
--url "https://gitlab.example.com/" \
--registration-token "REG_TOKEN" \
--name "runner-docker" \
--executor "docker" \
--docker-image "docker:24.0" \
--docker-privileged \
--docker-volumes "/cache"
Adjust config.toml for Docker and BuildKit
After registration, edit config.toml to enable BuildKit and set volumes. BuildKit speeds up builds and improves layer caching. We keep the executor privileged for scenarios that require DinD. To optimize your image layers, see guidance on creating lightweight images and cache strategies in the linked resources.
concurrent = 2
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "runner-docker"
url = "https://gitlab.example.com/"
token = "TOKEN_GERADO_NO_REGISTRO"
executor = "docker"
environment = [
"DOCKER_DRIVER=overlay2",
"DOCKER_TLS_CERTDIR=",
"DOCKER_BUILDKIT=1",
"BUILDKIT_PROGRESS=plain"
]
[runners.docker]
image = "docker:24.0"
privileged = true
tls_verify = false
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
Enable S3/MinIO remote cache for the Runner
Now configure the remote cache to restore and save dependencies faster. If you use MinIO without TLS, enable Insecure. Restrict bucket access to a dedicated user. Place the block below inside your [[runners]] section in config.toml and adjust endpoint, region, bucket and credentials for your provider.
[runners.cache]
Type = "s3"
Shared = true
[runners.cache.s3]
ServerAddress = "minio.internal:9000"
BucketName = "gitlab-cache"
BucketLocation = "us-east-1"
AccessKey = "MINIO_ACCESS_KEY"
SecretKey = "MINIO_SECRET_KEY"
Insecure = true
Pipeline (.gitlab-ci.yml) with BuildKit and registry cache
This example builds and publishes the image to the GitLab Container Registry using BuildKit with cache stored in the registry. That reuses layers between builds, reducing time for repeated Dockerfile steps. The docker:dind service exposes the daemon on 2375 without TLS. Use this only in controlled environments. Registry credentials come from standard GitLab CI variables.
image: docker:24.0
services:
- name: docker:24.0-dind
command: ["--registry-mirror=", "--mtu=1460"]
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_TLS_CERTDIR: ""
DOCKER_DRIVER: overlay2
DOCKER_BUILDKIT: "1"
BUILDKIT_PROGRESS: plain
IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
CACHE_REF: $CI_REGISTRY_IMAGE:buildcache
stages: [build]
build-image:
stage: build
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
- docker buildx create --use --name builder || docker buildx use builder
script:
- docker buildx build \
--push -t "$IMAGE" \
--cache-from type=registry,ref="$CACHE_REF" \
--cache-to type=registry,ref="$CACHE_REF",mode=max \
-f Dockerfile .
rules:
- if: $CI_COMMIT_BRANCH
Verification and common troubleshooting
Run the commands below to validate logs and connectivity. The first build may take longer; subsequent builds should fetch fewer dependencies and skip Dockerfile steps that were cached.
- If the runner doesn't show up in GitLab, check the token and URL.
- If the S3 cache isn't working, validate endpoint, keys and bucket policy.
- If buildx fails, check Docker and dind service versions.
# Ver logs do runner
sudo docker logs --tail=200 gitlab-runner
# Verificar conectividade e estado do runner
sudo docker exec -it gitlab-runner gitlab-runner verify --delete
# Conferir se o bloco de cache está ativo no config
echo "--- cache config ---" && \
sudo docker exec -it gitlab-runner \
grep -A4 "\[runners.cache\]" /etc/gitlab-runner/config.toml
Operation, security and maintenance
Keep the runner up to date and sized to your infrastructure. Aggressive cleanups can remove useful layers and caches — schedule them carefully. For host sizing and monitoring guidance, see the linked materials.
- Update the runner container periodically and redeploy using the same config volume.
- Limit concurrency (concurrent) to what the host can handle. Sizing guidance: How to choose the right VPS size.
- Monitor CPU, RAM and disk. Instrument with Prometheus & Grafana via Docker Compose.
- Be careful with prune: it can remove images and caches; run it outside peak hours.
# Atualizar imagem do runner e inspecionar estado
sudo docker pull gitlab/gitlab-runner:alpine
sudo docker exec -it gitlab-runner gitlab-runner list
sudo docker exec -it gitlab-runner gitlab-runner status
# Limpeza com cautela (pode remover camadas e caches)
sudo docker system prune -af --volumes
Frequently asked questions
Do I need Docker-in-Docker or the host socket?
For the Docker executor it's common to mount the host socket into the runner container. This avoids DinD overhead and is usually simpler. Use DinD when you want a fully isolated Docker daemon per job, accepting higher resource usage.
What's the difference between Runner cache and Docker (BuildKit) cache?
The Runner cache stores job files like language dependencies. The Docker cache stores image layers created during builds. They complement each other: the first speeds application tasks, the second speeds Dockerfile rebuilds.
Where are Runner cache files stored?
With S3/MinIO configured, cache artifacts are uploaded to the specified bucket. Without S3, cache stays local on the runner host, which makes sharing harder and risks loss when the container is recreated.
How do I limit concurrent jobs?
Set the concurrent key in config.toml to define the maximum simultaneous jobs per runner service. Balance it with server capacity. Monitor CPU, memory and I/O to find the right level without causing contention.
Can I share the same cache across different projects?
Yes, if you set Shared = true in the runner cache block and use compatible cache keys in jobs. Avoid generic keys to prevent data leaks. Separate buckets or key prefixes help isolate projects.
How to rotate S3/MinIO credentials securely?
Create a dedicated user with minimal permissions for the bucket and rotate keys regularly. Never commit credentials. Use environment variables in the runner container or your platform's secret management where possible.
Want to speed up your CI securely?
If you need help designing a self-hosted runner, configuring remote cache and optimizing Dockerfiles and pipelines, we can discuss your setup and provide an assessment.
Talk about your CI/CDSee also
- Tutorial: Implement Canary releases with Traefik in Docker Compose using headers and weighted routing for gradual PHP deployments
- Tutorial: Configure HAProxy with Let's Encrypt and health checks to load balance traffic between Docker containers on a VPS (step-by-step)
- Hands‑On Tutorial: Implement mTLS Between Nginx and Microservices with smallstep CA in Docker Compose
- Tutorial: configure cgroup v2 and systemd ResourceControl to limit CPU, memory and I/O for Docker containers on Debian