Jenkins CI/CD Pipelines

Architecture

Jenkins has a controller (historically called the master) and one or more agents. The controller stores configuration, schedules work, serves the web UI and keeps build history. Agents are machines, VMs or containers that execute the actual build steps. Agents connect by SSH or by launching an inbound agent process, and you target them with labels.

• Run builds on agents, not on the controller. Set the controller's executor count to zero in production.
• Executors are slots on a node; the number of executors limits concurrent builds there.
• Almost everything is a plugin: Git, Pipeline, credentials, role-based access, notifications. Keep the plugin set small and updated.
• All state lives under JENKINS_HOME. Back it up, or better, define configuration as code with the Configuration as Code (JCasC) plugin so a controller can be rebuilt from a YAML file.

Ephemeral agents (a fresh container or VM per build) give clean, reproducible environments and are the modern default.

Jobs versus pipelines

A freestyle job is configured through the UI: pick a source repository, add build steps, add post-build actions. It is quick to start but the configuration lives only in Jenkins, is hard to review, and does not scale to multi-stage delivery.

A pipeline describes the whole flow as code in a Jenkinsfile stored with the application. That gives you version history, code review, branch-specific behaviour and the ability to survive a controller restart. A multibranch pipeline scans a repository and creates a job per branch or pull request automatically. Prefer pipelines for anything beyond a quick experiment.

Both job types can be parameterised: string, boolean and choice parameters let users supply input such as a target environment when starting a build.

The Jenkinsfile

Declarative syntax is the recommended style: a fixed structure that is easy to read and validate.

pipeline {
  agent { label 'linux' }
  options { timeout(time: 30, unit: 'MINUTES') }
  parameters {
    choice(name: 'TARGET', choices: ['staging', 'production'], description: 'Deploy target')
  }
  environment { APP_NAME = 'demo-app' }
  stages {
    stage('Build') {
      steps { sh 'mvn -B clean package' }
    }
    stage('Test') {
      steps { sh 'mvn -B test' }
      post { always { junit 'target/surefire-reports/*.xml' } }
    }
    stage('Deploy') {
      when { branch 'main' }
      steps {
        retry(2) { sh "./deploy.sh ${params.TARGET}" }
      }
    }
  }
  post {
    failure { echo 'Build failed' }
    cleanup { cleanWs() }
  }
}

• agent chooses where to run: a label, any, or a Docker image.
• stages/steps hold the work; parallel runs stages concurrently.
• when makes a stage conditional; post runs after the stage or pipeline depending on the outcome.
• retry, timeout and input (manual approval) are built-in controls.
• Scripted pipelines use full Groovy and are more flexible but harder to maintain; use a script block sparingly inside declarative code.
• Shared libraries hold reusable Groovy functions across many repositories.

Credentials handling

Jenkins has a credentials store that supports secret text, username and password, SSH private keys, certificates and secret files. Each entry has an ID, and jobs reference the ID rather than the value. Secrets are encrypted at rest and masked in console output.

withCredentials([
  usernamePassword(credentialsId: 'registry-login',
                   usernameVariable: 'REG_USER',
                   passwordVariable: 'REG_PASS'),
  string(credentialsId: 'api-token', variable: 'API_TOKEN')
]) {
  sh 'echo "$REG_PASS" | docker login registry.example.com -u "$REG_USER" --password-stdin'
}

Further habits: scope credentials to a folder rather than globally, prefer short-lived tokens or an external secret manager over long-lived static keys, rotate anything that was ever printed, and grant cloud access through instance roles where possible.

Triggers

• Webhook: the repository host notifies Jenkins on push or pull request. This is the preferred approach because it is immediate and cheap.
• Poll SCM: Jenkins checks the repository on a schedule and builds when something changed. Simple behind a firewall, but wasteful.
• Cron (build periodically): time-based, for nightly builds. The five fields are minute, hour, day of month, month and day of week. Use the H symbol (for example H 2 * * *) to spread load instead of everything firing at the same minute.
• Upstream: run after another job finishes.
• Remote trigger: call the job's build URL with an authentication token, for scripts and external systems.

Pipelines declare these in a triggers { } block, so they are versioned with the code.

SCM, Maven and Ansible

Source control

Install the Git plugin, add a repository credential, and use checkout scm (automatic in multibranch jobs). Use SSH deploy keys or tokens with read-only scope for checkout.

Maven

Configure a JDK and Maven under tool settings, then call mvn -B clean package in a stage. Publish test results with junit and keep the resulting JAR with archiveArtifacts. A common flow is build, unit test, package, then deploy the artifact to a repository or a server.

Ansible

Jenkins orchestrates, Ansible configures. Install Ansible on the agent (or use an agent image that has it), fetch an SSH key through credentials, and run the playbook with parameters passed as extra variables:

sh 'ansible-playbook -i inventory/${TARGET}.ini site.yml --extra-vars "app_version=${APP_VERSION}"'

The Ansible plugin can wrap this and colourise output. Remote shell commands over SSH are also possible via the SSH plugins, but keep real logic in playbooks or scripts that live in version control.

Security

• Authentication: never leave the instance open. Use a security realm backed by the Jenkins user database, LDAP or single sign-on, and disable anonymous access and public sign-up unless you truly want it.
• Authorization: the Role-based Authorization Strategy plugin defines global and project roles (admin, developer, read-only) and assigns users or groups to them. Matrix authorization is the alternative.
• Keep CSRF protection enabled (it is on by default in current releases), put Jenkins behind HTTPS, and keep core and plugins patched, since plugins are the most common source of vulnerabilities.
• Restrict who can edit Jenkinsfiles and shared libraries, because they run arbitrary code on your agents.
• Isolate agents, avoid running builds as root, and keep production credentials out of jobs that build untrusted pull requests.
• Email or chat notifications on failure, via the post section, keep teams informed without anyone watching the console.