Jenkins: the controller, agents, Jenkinsfile and shared libraries

Why learn Jenkins

Jenkins is the open-source automation server most companies started their CI on, and many still run it: you host it yourself, it connects to any Git server, and thousands of plugins extend it. You'll meet it in banks, telecoms and any team with years of pipelines behind it. The ideas are the ones you already know; the spelling is Groovy.

The controller and agents

⤢ View

The controller plans and stores; agents build. Keep builds off the controller, where every credential lives.© Diagram: DevOps Academy

Close

Set the controller's number of executors to 0 (Manage Jenkins → Nodes → Built-In Node). A build on the controller runs with access to JENKINS_HOME, where every job, credential and secret is stored. With the Kubernetes plugin, each build gets a fresh pod that's deleted afterwards: the same "clean machine every time" that GitHub-hosted runners give you.

The Jenkinsfile

A Jenkinsfile in the repository defines the pipeline as code, like .github/workflows/. Use the declarative syntax: structured and validated. Here's shop-api's pipeline from the earlier lessons:

// Jenkinsfile
pipeline {
  agent { docker { image 'node:22-alpine' } }        // every stage runs in this container

  options {
    timeout(time: 20, unit: 'MINUTES')
    disableConcurrentBuilds()
    buildDiscarder(logRotator(numToKeepStr: '30'))
  }

  environment {
    VERSION = "1.0.${env.BUILD_NUMBER}-${env.GIT_COMMIT.take(7)}"
  }

  stages {
    stage('Install') {
      steps { sh 'npm ci' }
    }
    stage('Checks') {
      parallel {                                       // lint and unit at the same time
        stage('Lint') { steps { sh 'npm run lint' } }
        stage('Unit') { steps { sh 'npm run test:coverage' } }
      }
    }
    stage('Integration') {
      steps { sh 'npm run test:integration' }
    }
    stage('Package') {
      steps {
        sh 'mkdir -p dist && cp -r src package.json dist/ && tar -czf shop-api-${VERSION}.tgz -C dist .'
        archiveArtifacts artifacts: 'shop-api-*.tgz', fingerprint: true
      }
    }
    stage('Deploy to production') {
      when { branch 'main' }
      input { message 'Deploy to production?'; ok 'Deploy' }   // waits before taking an agent
      agent { label 'deploy' }                         // an agent that can reach the servers
      steps {
        withCredentials([sshUserPrivateKey(credentialsId: 'deploy-key', keyFileVariable: 'SSH_KEY')]) {
          sh './scripts/deploy-ssh.sh deploy@shop.example.com "$VERSION" shop-api-*.tgz'
        }
      }
    }
  }

  post {
    failure { echo "Build ${env.BUILD_NUMBER} failed: ${env.BUILD_URL}" }
    always  { cleanWs() }
  }
}

The docker agent needs the Docker Pipeline plugin and agents with Docker; cleanWs() comes from the Workspace Cleanup plugin. Note the quotes: Groovy expands "${VERSION}" in double quotes itself, while in sh '…$VERSION…' (single quotes) the shell does, which is the safe way to pass values, for the same reason as env: in Actions (lesson 11).

Multibranch pipelines find every branch and pull request with a Jenkinsfile and build each one, and a webhook from GitHub or GitLab triggers them on push. The job's Pipeline Syntax page generates the code for any step from a form.

Plugins: power and risk

Almost everything in Jenkins is a plugin: Git, credentials, pipelines, Docker, Kubernetes agents, Slack messages. That flexibility is also its weakest point:

• Each plugin is maintained separately, and several have had serious vulnerabilities over the years.
• Plugins depend on each other and on the Jenkins version, so upgrades can break pipelines.
• They all run inside the controller, with its access to every credential.

Install only what you use, update Jenkins and plugins regularly (test upgrades on a copy first), and remove plugins nobody needs. Follow the official security advisories (linked below). Manage the controller itself as code: the Configuration as Code plugin describes its settings in YAML, and a Dockerfile can pin the plugin list, so a broken controller can be rebuilt instead of repaired by hand.

Shared libraries

With fifty services, fifty copies of the same Jenkinsfile drift apart, just like workflow files (lesson 12). A shared library is a Git repository of pipeline code that Jenkinsfiles load:

jenkins-shared-library/
└── vars/
    └── nodePipeline.groovy      # becomes a step called nodePipeline

// vars/nodePipeline.groovy
def call(Map config = [:]) {
  pipeline {
    agent { docker { image "node:${config.node ?: '22'}-alpine" } }
    stages {
      stage('Install') { steps { sh 'npm ci' } }
      stage('Lint')    { steps { sh 'npm run lint' } }
      stage('Test')    { steps { sh 'npm run test:coverage' } }
    }
    post { always { cleanWs() } }
  }
}

// Jenkinsfile in each service
@Library('company-pipelines@v1') _
nodePipeline(node: '22')

Register the library once under Manage Jenkins → System → Global Pipeline Libraries, pin it to a tag such as @v1, and version it like any shared code.

Jenkins or GitHub Actions?

Plenty of companies run both: Jenkins for legacy and on-premises builds, Actions for new services. The pipeline shape you've built in this course works in either.

$ docker volume create jenkins_home
$ docker run -d --name jenkins -p 8080:8080 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts-jdk21
$ docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword   # paste into http://localhost:8080