GoCD: Complete Configuration Guide

Introduction to GoCD

GoCD is an open-source continuous delivery server designed to model complex software delivery workflows. Unlike simpler CI tools, GoCD emphasizes end-to-end visibility through its value stream map, and supports advanced pipeline modeling with fan-in, fan-out, and dependency management. A complete understanding of its configuration is essential to harness these capabilities.

Why it matters: Modern delivery pipelines involve multiple teams, services, and environments. GoCD's configuration lets you define exactly how changes flow from code commit to production, enabling traceability, reproducibility, and controlled promotion. Mastering its configuration unlocks true continuous delivery.

Core Configuration Concepts

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

GoCD configuration is stored in a central XML file (or multiple JSON files in newer versions) that defines pipelines, stages, jobs, tasks, agents, environments, and more. The main concepts are:

Pipelines: The top-level unit of delivery, composed of stages.
Stages: Ordered phases within a pipeline (e.g., build, test, deploy). Each stage consists of jobs.
Jobs: Independent units of execution that run in parallel on agents. A job contains tasks.
Tasks: The actual commands or actions (e.g., shell scripts, Ant, Maven, Docker).
Materials: Triggers that start a pipeline, like Git repositories, other pipelines, or manual triggers.
Agents: Worker nodes that execute jobs. They can be assigned to environments.
Environments: Logical groupings of agents and pipelines to isolate production paths.
Artifacts: Files generated by jobs that can be consumed by downstream stages or pipelines.

Configuring Your First Pipeline

Let’s create a simple pipeline with one stage and one job that checks out code and runs a build script. In traditional XML configuration, you would define this in cruise-config.xml:

<?xml version="1.0" encoding="utf-8"?>
<cruise xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="cruise-config.xsd">
  <server>
    <security>
      <!-- Authentication and authorization -->
    </security>
  </server>
  <pipelines>
    <pipeline name="my-app-build">
      <materials>
        <git url="https://github.com/example/my-app.git"
             branch="main"
             materialName="my-app-repo" />
      </materials>
      <stage name="build" cleanWorkingDir="false">
        <job name="compile">
          <tasks>
            <exec command="make">
              <arg>clean</arg>
              <arg>all</arg>
              <runIf status="passed" />
            </exec>
          </tasks>
          <artifacts>
            <artifact src="target/*.jar" dest="dist" />
          </artifacts>
        </job>
      </stage>
    </pipeline>
  </pipelines>
  <environments>
    <environment name="development">
      <agents>
        <physical uuid="agent-1-uuid" />
      </agents>
    </environment>
  </environments>
</cruise>

This example defines a pipeline triggered by changes to the main branch. The single stage "build" contains a job "compile" that executes make clean all and collects JAR artifacts. The pipeline uses an agent assigned to the "development" environment.

In newer GoCD versions, you can also define pipelines using JSON configuration plugins or the GoCD API. For instance, a JSON equivalent might look like:

{
  "name": "my-app-build",
  "materials": [
    {
      "type": "git",
      "url": "https://github.com/example/my-app.git",
      "branch": "main",
      "name": "my-app-repo"
    }
  ],
  "stages": [
    {
      "name": "build",
      "cleanWorkingDir": false,
      "jobs": [
        {
          "name": "compile",
          "tasks": [
            {
              "type": "exec",
              "command": "make",
              "arguments": ["clean", "all"],
              "runIf": ["passed"]
            }
          ],
          "artifacts": [
            {
              "source": "target/*.jar",
              "destination": "dist"
            }
          ]
        }
      ]
    }
  ]
}

This JSON can be used with the GoCD configuration repository or the API to create pipelines programmatically.

Advanced Pipeline Modeling

GoCD shines when modeling complex workflows. You can connect pipelines as materials, enabling fan-in (multiple upstream pipelines trigger a downstream one) and fan-out (one pipeline triggers multiple downstream pipelines). This creates a value stream map that visualizes the entire delivery flow.

Example: A deployment pipeline that depends on two independent build pipelines:

<pipeline name="deploy-to-staging">
  <materials>
    <pipeline pipelineName="service-a-build" stageName="build" />
    <pipeline pipelineName="service-b-build" stageName="build" />
  </materials>
  <stage name="deploy">
    <job name="deploy-scripts">
      <tasks>
        <fetchPipeline pipelineName="service-a-build"
                       stageName="build"
                       jobName="compile"
                       src="dist"
                       dest="artifacts/service-a" />
        <fetchPipeline pipelineName="service-b-build"
                       stageName="build"
                       jobName="compile"
                       src="dist"
                       dest="artifacts/service-b" />
        <exec command="deploy.sh" />
      </tasks>
    </job>
  </stage>
</pipeline>

Here, the deploy pipeline automatically triggers when both service builds complete successfully. It fetches artifacts from each upstream pipeline and then executes deployment. This pattern ensures consistent, coordinated releases.

Managing Environments and Agents

Environments group pipelines and agents to separate concerns, like development, staging, and production. Agents are registered by their UUID and can be assigned to environments. You can also set environment variables scoped to an environment.

Example environment definition:

<environment name="production">
  <agents>
    <physical uuid="prod-agent-1" />
    <physical uuid="prod-agent-2" />
  </agents>
  <pipelines>
    <pipeline name="deploy-to-prod" />
  </pipelines>
  <environmentvariables>
    <variable name="TARGET_CLOUD">
      <value>aws</value>
    </variable>
  </environmentvariables>
</environment>

This restricts the production pipeline to only run on specified agents, and injects the TARGET_CLOUD variable into all jobs in that environment.

Using Configuration Repositories

GoCD supports configuration as code through config repositories. Instead of a monolithic XML file, you can store pipeline definitions in a Git repository, enabling team ownership, pull request reviews, and versioned configuration history. The server polls these repos and merges definitions.

To set up a config repo, define it in the server configuration:

<config-repos>
  <config-repo id="team-alpha-pipelines">
    <git url="https://github.com/my-org/gocd-pipelines.git" />
  </config-repo>
</config-repos>

Inside that Git repo, you place JSON or YAML pipeline files (depending on plugin). For example, a file pipelines/frontend.yaml (with appropriate plugin) can define a pipeline. This approach scales configuration across large organizations.

Best Practices for GoCD Configuration

Keep pipelines modular and single-responsibility: A pipeline should represent a coherent delivery unit (e.g., build, test, deploy) rather than a monolith of many unrelated tasks.
Use templates for common patterns: If many pipelines share the same stage structure, define a pipeline template and instantiate it. This reduces duplication and ensures consistency.
Version control your configuration: Always store pipeline definitions in a config repository and enforce code review. Use the GoCD API to test configuration changes in a staging server.
Leverage artifacts and external storage: Pass artifacts between stages and pipelines instead of rebuilding. Use the fetchArtifact task to promote binaries, ensuring they are built once and deployed consistently.
Secure secrets properly: Do not hardcode credentials in tasks. Use GoCD’s secure environment variables or a secret management plugin (e.g., HashiCorp Vault). These are encrypted at rest and masked in logs.
Design for fan-in and fan-out: Model dependencies explicitly. Use pipeline materials to visualize and enforce the flow, making value stream maps accurate and actionable.
Monitor and scale agents: Register agents with appropriate environments and resources. Use elastic agent plugins to auto-scale based on demand, keeping pipelines efficient.
Test configuration changes locally: Use the GoCD server’s config validation endpoint or run a local server with Docker to validate your XML/JSON before pushing.

Conclusion

GoCD’s configuration model provides the flexibility needed for modern continuous delivery. By understanding pipelines, stages, jobs, materials, and environments, you can craft workflows that accurately reflect your team’s delivery process. Adopting best practices like configuration as code, modular templates, and secure secret management ensures your pipelines are maintainable, scalable, and reliable. With the examples and concepts in this guide, you have a complete foundation to configure GoCD for any delivery scenario.