GitLab CI basics: Git & GitHub best practices explained

Introduction

We've spent weeks mastering Git fundamentals and GitHub workflows, but when your team scales beyond a handful of developers, you need automation that catches issues before code reaches production. GitLab CI basics: Git & GitHub best practices explained covers the pipeline system that makes this possible—the declarative workflows defined in .gitlab-ci.yml that turn every push into a potential release candidate. Whether you're migrating from GitHub Actions or building your first CI/CD system, understanding stages, jobs, runners, and artifacts is non-negotiable for professional development teams.

Stages and Jobs: Structuring Your Pipeline

A GitLab CI pipeline is built on two core concepts: stages and jobs. Stages are sequential phases of your pipeline—think build, test, deploy—that run one after another. Jobs are individual tasks that run within a stage, and multiple jobs in the same stage execute in parallel on different runners.

Let's say we're working on a Node.js application. We need to install dependencies, run unit tests, lint the code, and generate a build artifact. Here's how we structure this:

stages:
  - build
  - test
  - deploy

install_dependencies:
  stage: build
  script:
    - npm install
  artifacts:
    paths:
      - node_modules/

unit_tests:
  stage: test
  script:
    - npm run test
  dependencies:
    - install_dependencies

lint_code:
  stage: test
  script:
    - npm run lint

deploy_production:
  stage: deploy
  script:
    - npm run build
    - aws s3 sync dist/ s3://my-bucket/
  only:
    - main

Notice that unit_tests and lint_code both run in the test stage—they execute in parallel. The install_dependencies job in the build stage completes first, then both test jobs start simultaneously. The deploy_production job won't start until all jobs in the test stage pass.

In production systems, we often have five to ten stages: compile, unit tests, integration tests, security scanning, staging deployment, and production deployment. This linear structure forces discipline—you can't jump straight to deployment without passing all earlier gates.

Runners Execute Jobs

A job definition is just configuration. The actual execution happens on runners—machines (physical or virtual) that listen to your GitLab instance for work. GitLab provides shared runners for free accounts, but production teams typically run dedicated runners on their own infrastructure.

Runners are tagged with labels describing their capabilities. You might have runners labeled docker, kubernetes, macos, or performance-test. When a job specifies which tags it needs, GitLab routes it to an appropriate runner:

build_macos_app:
  stage: build
  tags:
    - macos
    - xcode-14
  script:
    - xcodebuild -scheme MyApp -configuration Release

integration_test:
  stage: test
  tags:
    - docker
    - large-memory
  script:
    - docker-compose up
    - pytest tests/integration/

The build_macos_app job only runs on a runner with both the macos and xcode-14 tags. If no such runner exists or all are busy, the job queues. The integration_test job, meanwhile, runs on a Docker-capable runner with enough memory.

In our experience, teams underestimate how much infrastructure management runners require. Each runner needs monitoring, updates, and disk cleanup. For teams under fifty developers, GitLab's SaaS offering with shared runners often beats self-hosting. For larger teams or those with strict data residency requirements, dedicated runners become necessary—budget accordingly.

Caching and Artifacts: Managing Pipeline Data

Pipelines generate data. Dependencies get downloaded, compiled objects get created, test reports get generated. Without intelligent caching and artifact management, pipelines become slow and storage bloats.

Cache is for speeding up jobs. It's temporary storage shared across pipeline runs on the same runner. The classic use case is dependency directories:

install_dependencies:
  stage: build
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
      - .npm/
  script:
    - npm ci --cache .npm --prefer-offline

test_job:
  stage: test
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/
  script:
    - npm test

The first job installs dependencies and caches node_modules/. The second job restores that cache—no reinstalling. The cache key uses the branch slug, so different branches have separate caches. Cache is ephemeral; GitLab can delete it without warning if storage limits are reached.

Artifacts are permanent outputs you want to keep. Build results, Docker images, test reports, coverage reports—these are artifacts. They persist after the pipeline completes:

build_artifact:
  stage: build
  script:
    - npm run build
    - tar -czf dist.tar.gz dist/
  artifacts:
    paths:
      - dist.tar.gz
    expire_in: 30 days

test_job:
  stage: test
  script:
    - npm test -- --coverage
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml
    paths:
      - coverage/
    expire_in: 90 days

The build_artifact job creates a tarball and stores it as an artifact for 30 days. The test_job publishes coverage reports that GitLab displays in the merge request UI, and stores the raw coverage directory for 90 days. Developers can download artifacts from the pipeline UI or API, useful for debugging or deploying specific builds.

A common pitfall: treating artifacts as permanent storage. Set expiration times based on your team's actual needs. Storing every artifact forever will exceed your GitLab quota within months.

Merge Request Pipelines

CI becomes truly valuable when it gates code review. Merge request pipelines run your pipeline against every pull request, letting reviewers see test results, coverage changes, and build status before merging.

In GitLab, you configure which pipelines run on merge requests using rules:

stages:
  - test
  - build
  - deploy

test:
  stage: test
  script:
    - npm test
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    - if: '$CI_COMMIT_BRANCH == "main"'

build:
  stage: build
  script:
    - npm run build
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "main"'

deploy:
  stage: deploy
  script:
    - npm run deploy
  rules:
    - if: '$CI_COMMIT_BRANCH == "main"'
      when: manual

The test job runs on every merge request and every push to main. The build job runs on main directly and on merge requests targeting main. The deploy job only runs on main, but requires manual approval (useful for production deploys).

Require all pipelines to pass before merging—this is enforced in GitLab under "Protected Branches". A developer creates a branch, pushes code, opens a merge request, and can't merge until the pipeline passes. This straightforward gating prevents a staggering amount of production damage.

In practice, we make merge request pipelines fast. Full test suites can take 15+ minutes; if developers wait that long before checking their code, they'll lose focus. Split expensive tests into separate pipelines that run asynchronously. Run smoke tests and unit tests synchronously (should complete in under 5 minutes), integration and performance tests async.

Common Mistakes

• No cache strategy. Teams often reinstall dependencies or recompile code on every job, making pipelines glacially slow. Invest time in caching logic early.
• Unclear artifact retention. Artifacts accumulate unchecked, consuming quota and making backups expensive. Always set expire_in and review retention policies quarterly.
• Mixing CI and CD. Deploying to production on every merge request to main is dangerous. Use environment-specific rules and require manual approval for production.
• Insufficient runner capacity. As your team grows, shared runners become bottlenecks. Plan for dedicated runners before developers complain about queue times.
• Ignoring pipeline metrics. Track average pipeline duration, failure rates by stage, and runner utilization. Pipeline performance degrades gradually; metrics surface problems early.

Key Takeaways

• Structure pipelines with stages (sequential) and jobs (parallel within a stage) to balance speed and clarity.
• Runners are the execution layer—tag them with capabilities and route jobs accordingly; plan for dedicated runners in production environments.
• Cache for speed (dependencies, build artifacts), use artifacts for permanent outputs (test reports, compiled binaries) with appropriate expiration.
• Gate code review with merge request pipelines and require all jobs to pass before merging; this is your first line of defense against buggy code.
• Optimize merge request pipelines for speed (aim for under 5 minutes) and run expensive tests asynchronously.
• Monitor pipeline metrics continuously—duration, success rates, runner utilization—to identify bottlenecks before they frustrate the team.