What is a CI/CD Pipeline?
A CI/CD pipeline is an automated workflow that takes your code from version control through a series of stages — building, testing, and deploying — without requiring manual intervention at each step. CI stands for Continuous Integration, which means automatically integrating code changes from multiple developers into a shared repository, running builds and tests to catch issues early. CD can mean either Continuous Delivery (automatically preparing code for release) or Continuous Deployment (automatically pushing changes to production). Together, they form the backbone of modern software delivery.
What Are GitHub Actions?
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →GitHub Actions is GitHub's native CI/CD platform, deeply integrated into every repository. It allows you to define event-driven workflows — written as YAML files — that trigger on events like pushes, pull requests, issue comments, or scheduled cron jobs. Each workflow consists of one or more jobs, which run in parallel or sequentially on hosted runners (Linux, macOS, or Windows) or self-hosted machines. Each job contains a series of steps that execute shell commands, scripts, or reusable community-built actions from the GitHub Marketplace.
Why CI/CD Matters
- Early bug detection: Automated tests run on every push, catching regressions before they reach production.
- Faster feedback loops: Developers receive immediate notifications when builds break, reducing debugging time.
- Consistent environments: Every build runs in an isolated, reproducible container or virtual machine.
- Reduced manual toil: Repetitive tasks like linting, testing, and deployment are fully automated.
- Auditability: Every workflow run is logged and traceable, making compliance and debugging straightforward.
- Team confidence: Merging pull requests becomes safe because you know the pipeline has already validated the changes.
Understanding the Workflow File Structure
All GitHub Actions workflows live in the .github/workflows/ directory at the root of your repository. Each workflow is a YAML file with a .yml extension. Let's break down the anatomy of a workflow file piece by piece.
The Name and Trigger Events
The top-level name property is optional but recommended — it appears in the Actions tab of your repository. The on key defines which events trigger the workflow. You can specify a single event, an array of events, or even activity types for more granular control.
name: CI Pipeline
on:
push:
branches: [main, develop]
pull_request:
branches: [main]
workflow_dispatch: # allows manual triggering from the UI
Jobs, Runners, and Steps
Under the jobs key, you define one or more named jobs. Each job specifies a runner OS via runs-on and a list of steps. Steps can be shell commands using run or references to community actions using uses.
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: npm ci
- name: Run tests
run: npm test
Building Your First CI/CD Pipeline: A Step-by-Step Guide
We'll build a complete pipeline for a Node.js application that includes linting, testing, building a Docker image, and deploying to a staging environment. Each stage builds on the previous one, giving you a production-ready workflow by the end.
Step 1: Create the Workflow Directory
At the root of your repository, create the directory structure:
mkdir -p .github/workflows
Step 2: Scaffold the Base Workflow File
Create a file named ci-cd.yml inside .github/workflows/ with the following skeleton. This workflow triggers on pushes and pull requests to main.
name: Full CI/CD Pipeline
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
# We will fill this in step by step
Step 3: Add the Lint Job
The lint job runs ESLint to enforce code style. It checks out the code, sets up Node.js, installs dependencies, and runs the linter. This job runs independently and in parallel with other jobs we'll add later.
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
Step 4: Add the Test Job with a Matrix Strategy
Testing across multiple Node.js versions catches compatibility issues. A matrix strategy creates a job instance for each combination of variables you define. Below, we run tests on Node.js 18, 20, and 22 in parallel.
test:
needs: lint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: ['18', '20', '22']
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run unit tests
run: npm test
- name: Upload coverage report
if: matrix.node-version == '20'
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage/
Notice the needs: lint key — this makes the test job wait for lint to succeed before starting, saving compute time if linting fails. The if conditional on the upload step ensures we only upload coverage once, avoiding duplicates.
Step 5: Add the Build Job for Docker
This job builds a Docker image, tags it with the commit SHA, and pushes it to a container registry. We use GitHub Container Registry (ghcr.io) for convenience, but the pattern works for Docker Hub, AWS ECR, or any OCI-compatible registry.
build-and-push:
needs: test
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/my-app:${{ github.sha }}
ghcr.io/${{ github.repository_owner }}/my-app:latest
The permissions block grants the job access to read repository contents and write packages. The GITHUB_TOKEN secret is automatically provided by GitHub — no manual secret creation required for ghcr.io authentication.
Step 6: Add the Deploy Job
The final job deploys the newly built image to a staging environment. Here we simulate an SSH-based deployment to a remote server, using repository secrets to store sensitive credentials.
deploy-staging:
needs: build-and-push
runs-on: ubuntu-latest
environment: staging
steps:
- name: Deploy via SSH
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.STAGING_HOST }}
username: ${{ secrets.STAGING_USER }}
key: ${{ secrets.STAGING_SSH_KEY }}
script: |
docker pull ghcr.io/${{ github.repository_owner }}/my-app:${{ github.sha }}
docker compose -f docker-compose.staging.yml up -d --remove-orphans
docker system prune -f
The environment: staging reference ties this job to a GitHub Environment, which lets you require manual approval, enforce protection rules, and scope secrets to that environment. You create the environment in the repository settings under Environments.
The Complete Workflow File
Here is the entire workflow file assembled, ready to drop into .github/workflows/ci-cd.yml:
name: Full CI/CD Pipeline
on:
push:
branches: [main]
pull_request:
branches: [main]
workflow_dispatch:
jobs:
lint:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run linter
run: npm run lint
test:
needs: lint
runs-on: ubuntu-latest
strategy:
matrix:
node-version: ['18', '20', '22']
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run unit tests
run: npm test
- name: Upload coverage report
if: matrix.node-version == '20'
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage/
build-and-push:
needs: test
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Log in to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push Docker image
uses: docker/build-push-action@v6
with:
context: .
push: true
tags: |
ghcr.io/${{ github.repository_owner }}/my-app:${{ github.sha }}
ghcr.io/${{ github.repository_owner }}/my-app:latest
deploy-staging:
needs: build-and-push
runs-on: ubuntu-latest
environment: staging
steps:
- name: Deploy via SSH
uses: appleboy/ssh-action@v1.0.3
with:
host: ${{ secrets.STAGING_HOST }}
username: ${{ secrets.STAGING_USER }}
key: ${{ secrets.STAGING_SSH_KEY }}
script: |
docker pull ghcr.io/${{ github.repository_owner }}/my-app:${{ github.sha }}
docker compose -f docker-compose.staging.yml up -d --remove-orphans
docker system prune -f
Setting Up Repository Secrets and Environments
For the deploy job to work, you need to store sensitive values as encrypted secrets. Navigate to your repository on GitHub, go to Settings → Secrets and variables → Actions, and add the following repository secrets:
STAGING_HOST— the IP address or hostname of your staging serverSTAGING_USER— the SSH username (e.g.,deploy)STAGING_SSH_KEY— the private SSH key for authentication
Next, create the staging environment under Settings → Environments. Here you can optionally require manual approval before deployment, specify protected branches, or scope the secrets directly to the environment instead of the entire repository. Environment-scoped secrets are only accessible to jobs that reference that environment with the environment key, adding an extra layer of security.
Testing Your Pipeline
Push a commit to the main branch or open a pull request against it. Head to the Actions tab in your repository to watch the workflow run. You'll see each job's status — lint, the three parallel test matrix runs, the Docker build-and-push, and finally the deploy job. If any step fails, click into the job to see detailed logs. The workflow is also triggerable manually via the workflow_dispatch event — look for the "Run workflow" button in the Actions tab.
Practical Example: A Python Application Pipeline
GitHub Actions is language-agnostic. Here is a concise pipeline for a Python project using pytest and flake8, with deployment to PyPI when a release is published.
name: Python CI/CD
on:
push:
branches: [main]
release:
types: [published]
workflow_dispatch:
jobs:
lint-and-test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12']
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Lint with flake8
run: flake8 src/ tests/
- name: Test with pytest
run: pytest --cov=src --cov-report=xml
- name: Upload coverage to Codecov
if: matrix.python-version == '3.12'
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: coverage.xml
publish-pypi:
needs: lint-and-test
if: github.event_name == 'release' && github.event.action == 'published'
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Build package
run: |
pip install build
python -m build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Best Practices for GitHub Actions Pipelines
- Use
npm ciorpip install --sync: Always prefer locked dependency installation (cifor npm,--syncfor pip viauvorpip-tools) overinstallto ensure reproducible builds. Thecicommand respectspackage-lock.jsonexactly and fails if it's out of sync. - Cache dependencies aggressively: Use the built-in
cacheoption onsetup-node,setup-python, and similar actions. For custom caching, useactions/cache@v4to cachenode_modules,~/.cache/pip, or Docker layers. - Pin action versions: Reference actions with a specific version tag (e.g.,
@v4) or a commit SHA for critical security workflows. Avoid floating references like@mainin production pipelines. - Minimize secrets exposure: Never print secrets in logs. Use
secrets.CUSTOM_SECRETsyntax and avoid passing secrets into scripts that might echo them. Usemaskworkflow commands if you must derive secrets dynamically. - Use job
needsfor sequential stages: Chain jobs withneedsto fail fast — if linting fails, don't waste minutes running tests or building images. Use parallel jobs only for independent work like matrix tests. - Leverage environments for deployment: Create separate environments (staging, production) with protection rules. Require manual approval for production deployments and restrict which branches can deploy to each environment.
- Keep workflows DRY: Extract reusable logic into reusable workflows (stored in
.github/workflows/reusable-*.yml) or composite actions. Call them withuses: ./.github/workflows/reusable-test.ymlto avoid copy-pasting steps across workflows. - Monitor workflow runs: Enable notifications for workflow failures in your repository settings. Consider integrating with Slack, Discord, or email notifications via community actions like
slackapi/slack-github-action. - Secure your supply chain: Use
actions/attest-build-provenanceto generate signed build provenance attestations. Combine withcosignfor container image signing to ensure your artifacts are verifiable. - Clean up artifacts: Artifacts and container images accumulate quickly. Set up a scheduled workflow that prunes old artifacts using
actions/delete-artifactsor cleans up old container images from your registry.
Going Further: Advanced Patterns
Once you're comfortable with the basics, explore these advanced capabilities:
- Reusable workflows: Define a workflow in
.github/workflows/reusable-ci.ymlthat accepts inputs likenode-versionandrun-tests. Other workflows call it withusesand pass parameters viawithandsecrets: inherit. - Concurrency groups: Use
concurrencyat the workflow or job level to prevent multiple deployments from running simultaneously on the same environment, avoiding race conditions. - Self-hosted runners: Register your own machines as runners for access to private infrastructure, specialized hardware (GPUs), or larger storage. Label them and reference the label in
runs-on. - OpenID Connect (OIDC): Authenticate with cloud providers like AWS, Azure, or GCP without storing long-lived credentials. Configure the provider, then use
aws-actions/configure-aws-credentialswith OIDC tokens.
Conclusion
You have now built a complete, production-grade CI/CD pipeline using GitHub Actions — from linting and multi-version testing to Docker image publishing and secure deployment. The workflow file you created is fully functional, event-driven, and extensible. GitHub Actions removes the friction of managing separate CI/CD infrastructure by embedding automation directly into your development workflow. Start with a simple workflow, add stages incrementally, and apply the best practices outlined here to keep your pipelines fast, secure, and maintainable. The combination of deep GitHub integration, a rich marketplace ecosystem, and flexible YAML-based configuration makes it one of the most powerful and accessible CI/CD platforms available today.