Workflow details: expressions, matrices, caches and artifacts

Expressions and contexts

Anything inside ${{ … }} is an expression, evaluated by GitHub before the step runs. Expressions read contexts, objects describing the run:

Expressions also have functions: contains(), startsWith(), format(), toJSON(), hashFiles(), and the status checks below.

Passing values: steps and jobs

A step writes name=value to the file in $GITHUB_OUTPUT; later steps read it through the step's id. A job publishes step outputs as job outputs for the jobs that need it:

build:
  runs-on: ubuntu-latest
  outputs:
    version: ${{ steps.version.outputs.version }}     # 3. publish for other jobs
  steps:
    - id: version                                     # 1. give the step an id
      run: echo "version=1.0.${{ github.run_number }}-${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"
    - run: echo "Building ${{ steps.version.outputs.version }}"   # 2. read it in a later step

report:
  needs: build
  runs-on: ubuntu-latest
  steps:
    - run: echo "version ${{ needs.build.outputs.version }}"      # 4. read it in another job

$ echo "version=1.0.42-${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"   # what step 1 does, run locally
$ cat "$GITHUB_OUTPUT"
version=1.0.42-e3b0c44

export VAR=x doesn't survive the step: each run: is its own shell.

Conditions: if

- run: ./deploy.sh
  if: github.ref == 'refs/heads/main' && github.event_name == 'push'
- run: ./notify-team.sh "build failed"
  if: failure()                       # only when an earlier step failed
- run: ./upload-logs.sh
  if: always()                        # whatever happened

Inside if: the ${{ }} is optional. Use !cancelled() rather than always() for cleanup that shouldn't run when someone cancels a deploy on purpose.

Matrix builds

A matrix runs the same job once per combination of values, in parallel. One block tests every Node version you support:

⤢ View

Matrix: one job definition, many parallel jobs. Cache: faster installs. Artifact: this run's output, handed to the next job.© Animation: DevOps Academy

Close

test:
  name: test (node ${{ matrix.node }}, ${{ matrix.os }})
  runs-on: ${{ matrix.os }}
  strategy:
    fail-fast: false
    matrix:
      os: [ubuntu-latest]
      node: [20, 22, 24]
      include:
        - os: windows-latest     # one extra combination
          node: 22
  steps:
    - uses: actions/checkout@v7
    - uses: actions/setup-node@v7
      with:
        node-version: ${{ matrix.node }}
        cache: npm
    - run: npm ci
    - run: npm test

That's four jobs: Node 20, 22 and 24 on Ubuntu, plus Node 22 on Windows. exclude: removes combinations; max-parallel: limits how many run at once.

fail-fast defaults to true: the first failure cancels the rest. For tests, set it to false so every version reports, and you can see "only Node 20 fails" at a glance. Required status checks name each matrix job, so a clear name: keeps the branch protection settings readable.

Caching dependencies

Installing dependencies on every run is the slowest boring part of most pipelines. A cache saves a folder after one run and restores it in the next. For the common package managers, the setup actions do it for you:

- uses: actions/setup-node@v7
  with:
    node-version: 22
    cache: npm            # caches npm's download folder, keyed on package-lock.json
- run: npm ci             # installs exactly the lockfile, using the cache

For anything else, use actions/cache with a key you build:

- uses: actions/cache@v6
  with:
    path: ~/.cache/pip
    key: pip-${{ runner.os }}-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      pip-${{ runner.os }}-

Caches are per branch (a pull request can read caches from its base branch), unused ones are evicted after about a week, and repositories have a size limit. So a pipeline must still work with no cache: it's only ever a speed-up. Use npm ci, not npm install, in CI: it installs exactly what the lockfile says and fails if the lockfile and package.json disagree.

Artifacts: passing files between jobs

Jobs run on different machines, so the dist/ one job builds isn't there for the next. Upload it as an artifact:

build:
  steps:
    - run: tar -czf "shop-api-${{ steps.version.outputs.version }}.tgz" -C dist .
    - uses: actions/upload-artifact@v7
      with:
        name: shop-api
        path: shop-api-*.tgz
        retention-days: 7

report:
  needs: build
  steps:
    - uses: actions/download-artifact@v8
      with:
        name: shop-api           # the files land in the working folder

Artifacts also appear on the run's page for anyone with access to download: test reports, coverage, build logs. Each name can be uploaded once per run, so in a matrix give each job its own name, such as coverage-node-${{ matrix.node }}. Retention defaults to 90 days and counts against storage; keep it short for anything you rebuild anyway.

Concurrency: one run per branch

Push three commits to a pull request in a minute and you get three runs, though only the last one matters. Merge two pull requests close together and two deploys race each other to the same server. A concurrency group fixes both: GitHub allows one running and one pending run per group, and a newer pending run replaces the older one.

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}                 # one lane per workflow and branch
  cancel-in-progress: ${{ github.event_name == 'pull_request' }}  # PRs: kill the outdated run

A deploy workflow uses a fixed group name, such as deploy-production, so deploys from any branch or workflow wait in one queue. concurrency: also works on a single job, when only the deploy job needs the lock.

Path filters: run only when it matters

Fixing a typo in the README shouldn't run twenty minutes of tests. paths runs a workflow only when a matching file changed; paths-ignore skips it when only ignored files changed:

on:
  pull_request:
    paths-ignore:
      - 'docs/**'
      - '**/*.md'

You can't use both on the same event. A deploy workflow that tries:

name: deploy
on:
  push:
    branches: [main]
    paths: ['src/**', 'package*.json']
    paths-ignore: ['docs/**']

actionlint catches it:

$ actionlint
.github/workflows/deploy.yml:6:5: both "paths" and "paths-ignore" filters cannot be used for the same event "push". note: use '!' to negate patterns [events]
  |
6 |     paths-ignore: ['docs/**']
  |     ^~~~~~~~~~~~~

The fix is one list with a ! exception: paths: ['src/**', 'package*.json', '!src/**/*.md'].

One trap: when a path filter skips a workflow, its checks never report, and a pull request that requires them waits forever as Expected. Either don't make path-filtered workflows required, or keep the workflow always running and decide inside a job what to skip (lesson 12 does this for monorepos).

Service containers: a real database for tests

Integration tests want a real database, not mocks. services: starts containers next to the job, waits for their health check, and throws them away when the job ends, so every run starts empty:

integration:
  runs-on: ubuntu-latest
  services:
    postgres:
      image: postgres:16-alpine
      env:
        POSTGRES_USER: shop
        POSTGRES_PASSWORD: test-only        # a throwaway test password, not a secret
        POSTGRES_DB: shop_test
      ports:
        - 5432:5432
      options: >-
        --health-cmd "pg_isready -U shop -d shop_test"
        --health-interval 5s
        --health-timeout 5s
        --health-retries 10
  env:
    DATABASE_URL: postgres://shop:test-only@localhost:5432/shop_test
  steps:
    - uses: actions/checkout@v7
    - uses: actions/setup-node@v7
      with:
        node-version: 22
        cache: npm
    - run: npm ci
    - run: npm run test:integration

That is the same as running the container yourself, which is how to debug it locally:

$ docker run -d --name ci-postgres -e POSTGRES_USER=shop -e POSTGRES_PASSWORD=test-only -e POSTGRES_DB=shop_test \
    -p 5432:5432 --health-cmd "pg_isready -U shop -d shop_test" --health-interval 5s \
    --health-timeout 5s --health-retries 10 postgres:16-alpine
$ docker ps --filter name=ci-postgres --format '{{.Names}}\t{{.Status}}'
ci-postgres	Up 15 seconds (healthy)
$ docker run --rm --network host postgres:16-alpine psql "postgres://shop:test-only@localhost:5432/shop_test" \
    -c "select current_database(), current_user;"
 current_database | current_user
------------------+--------------
 shop_test        | shop
(1 row)
$ docker rm -f ci-postgres

When the job runs directly on the runner (as above), it reaches the service on localhost through the mapped port. When the job itself runs in a container (container: node:22), it reaches the service by its name instead, postgres:5432, with no ports: needed. Service containers need a Linux runner.

$ npm install                      # creates package-lock.json, which npm ci and the cache need
$ git add package-lock.json
$ nano .github/workflows/ci.yml    # the test matrix, build and report jobs from this lesson
$ actionlint && echo "workflow OK"
workflow OK
$ git commit -am "ci: test on a matrix, upload the build" && git push
$ gh run watch                     # follow the run
$ gh run download --name shop-api  # the artifact, on your machine
$ tar -tzf shop-api-*.tgz
./
./package.json
./src/
./src/server.js
./src/app.js