Lesson 10 · GitOps

GitOps: Git says what runs, the cluster makes it so

Deploy by changing Git, not by pushing to servers: push versus pull, a Kustomize config repo, Argo CD and Flux syncing it to Kubernetes, drift repair, and git revert rollbacks.

40 min read · intermediate · hands-on

By the end of this lesson you'll be able to

0 of 6 complete

Explain push and pull deployment, and what GitOps changes about access and auditing
Lay out a config repository with Kustomize, and change an image tag the way CI does
Define an Argo CD Application that syncs, prunes and self-heals
Define the same deployment in Flux with a GitRepository and a Kustomization
Connect CI to GitOps: build the image, then commit its tag to the config repository
Roll back with git revert, and see drift corrected automatically

Push and pull deploys

In the deploys so far, CI pushes: it holds credentials for the target and runs commands against it. With GitOps, nobody pushes to production. A Git repository describes what should run, and an agent inside the cluster pulls it and makes the cluster match:

Two deployment models. Push: CI builds the image and then connects to the cluster with kubectl and stored credentials to apply changes, so CI holds the keys to production. Pull, or GitOps: CI builds and pushes the image to a registry, then commits the new image tag to a config repository. Argo CD or Flux, running inside the cluster, watches that repository, notices the change and applies it. It also compares the cluster with Git continuously and repairs drift. A rollback is a git revert in the config repository. CI never touches the cluster. — Push: CI reaches into production. Pull: CI changes Git, and the cluster updates itself.© Diagram: DevOps Academy

	Push (CI applies)	Pull (GitOps)
Who changes the cluster	the pipeline, with cluster credentials	an agent inside the cluster
Credentials outside the cluster	yes, in CI secrets	none: CI only writes to Git
Audit trail	pipeline logs	Git history: who, what, when, approved by whom
Manual changes in the cluster	stay until the next deploy	undone automatically (self-heal)
Rollback	re-run an old pipeline	`git revert`

The OpenGitOps principles sum it up: the desired state is declarative, versioned in Git, pulled automatically, and continuously reconciled.

The config repository

GitOps needs a repository of manifests. A common layout keeps a shared base and one overlay per environment, with Kustomize (built into kubectl):

shop-config/
├── apps/shop-api/
│   ├── base/                  # what every environment shares
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   └── kustomization.yaml
│   └── production/
│       └── kustomization.yaml # production's namespace and image tag
└── clusters/production/
    └── argocd-shop-api.yaml   # tells Argo CD what to sync

The base is an ordinary Deployment and Service:

# apps/shop-api/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop-api
spec:
  replicas: 2
  selector:
    matchLabels: { app: shop-api }
  template:
    metadata:
      labels: { app: shop-api }
    spec:
      containers:
        - name: shop-api
          image: ghcr.io/elonerajeev/shop-api:1.0.0
          ports:
            - containerPort: 3000
          readinessProbe:
            httpGet: { path: /health, port: 3000 }
          resources:
            requests: { cpu: 50m, memory: 64Mi }
            limits: { memory: 128Mi }

# apps/shop-api/base/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: shop-api
spec:
  selector: { app: shop-api }
  ports:
    - port: 80
      targetPort: 3000

# apps/shop-api/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: [deployment.yaml, service.yaml]

The production overlay reuses the base and sets what's specific to production:

# apps/shop-api/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: shop
resources: [../base]
images:
  - name: ghcr.io/elonerajeev/shop-api
    newTag: 1.0.0

A release is one line changing. CI does it with Kustomize:

$ cd apps/shop-api/production
$ kustomize edit set image ghcr.io/elonerajeev/shop-api:1.0.43-9f2a1bc
$ grep -A2 '^images' kustomization.yaml
images:
- name: ghcr.io/elonerajeev/shop-api
  newTag: 1.0.43-9f2a1bc
$ kustomize build . | kubeconform -strict -summary -
Summary: 2 resources found parsing stdin - Valid: 2, Invalid: 0, Errors: 0, Skipped: 0

cd apps/shop-api/production

cdchange directory

grep -A2 '^images' kustomization.yaml

grepfind lines that match a pattern

kustomize build . | kubeconform -strict -summary -

.this folder
|pipe: send output to the next command
-the previous directory

kustomize build renders the final manifests, and kubeconform validates them against the Kubernetes schemas, so a typo fails in CI instead of in the cluster.

Argo CD

Argo CD runs in the cluster and syncs Applications: each one points at a path in a Git repository and a destination in the cluster. It has a web UI showing every resource, its health and its diff from Git.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: shop-api
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/elonerajeev/shop-config.git
    targetRevision: main
    path: apps/shop-api/production
  destination:
    server: https://kubernetes.default.svc     # the cluster Argo CD runs in
    namespace: shop
  syncPolicy:
    automated:
      prune: true           # delete what was removed from Git
      selfHeal: true        # undo manual changes in the cluster
    syncOptions:
      - CreateNamespace=true

Setting	Without it
`automated`	Argo CD shows OutOfSync and waits for someone to click Sync
`prune`	a manifest deleted from Git leaves its resource running
`selfHeal`	a `kubectl edit` in production stays until the next commit

The first Application is applied once with kubectl apply -f; after that, Argo CD can manage its own Applications from Git too (the "app of apps" pattern).

Flux

Flux does the same job as a set of small controllers with no central UI, configured entirely with Kubernetes resources: a source to watch, and what to apply from it:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: shop-config
  namespace: flux-system
spec:
  interval: 1m                # check Git every minute
  url: https://github.com/elonerajeev/shop-config
  ref:
    branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: shop-api
  namespace: flux-system
spec:
  interval: 5m                # re-apply and repair drift every 5 minutes
  sourceRef:
    kind: GitRepository
    name: shop-config
  path: ./apps/shop-api/production
  prune: true
  targetNamespace: shop
  wait: true                  # healthy only when the rollout finishes
  timeout: 2m

	Argo CD	Flux
Shape	one app with a rich web UI	small controllers, CLI-first
Unit	`Application` (and `ApplicationSet` for many)	`GitRepository` + `Kustomization` or `HelmRelease`
Multi-cluster	one Argo CD can manage many clusters	usually Flux in each cluster
Extras	UI, SSO, sync waves, rollouts	image automation that commits new tags for you

Both are CNCF graduated projects. Pick Argo CD when people want to see and click; pick Flux when everything, including Flux itself, should be plain YAML in Git. Both manifests above pass kubeconform with the CRD schemas from the community catalog, and a typo such as selfHeel is reported as additional properties 'selfHeel' not allowed.

CI and GitOps together

CI still builds, tests and publishes the image (lessons 5 and 6). Its last job is just a commit:

update-config:
  runs-on: ubuntu-latest
  environment: production                 # approval before production (lesson 4)
  steps:
    - uses: actions/checkout@v7
      with:
        repository: elonerajeev/shop-config
        token: ${{ secrets.CONFIG_REPO_TOKEN }}   # can write to the config repo only
    - name: Set the new image tag
      working-directory: apps/shop-api/production
      env:
        VERSION: ${{ inputs.version }}
      run: |
        [[ $VERSION =~ ^[0-9]+\.[0-9]+\.[0-9]+-[0-9a-f]{7}$ ]] || { echo "bad version: $VERSION"; exit 1; }
        kustomize edit set image "ghcr.io/elonerajeev/shop-api:$VERSION"
    - name: Commit and push
      env:
        VERSION: ${{ inputs.version }}
      run: |
        git config user.name "shop-api release bot"
        git config user.email "shop-api-bot@users.noreply.github.com"
        git commit -am "shop-api: deploy $VERSION to production"
        git push

The version check refuses anything that isn't a real tag: x; echo pwned and 1.0.43 are both rejected. The token can write to one repository and nothing else; the cluster's credentials never leave the cluster.

Rolling back

$ git log --oneline -2
f49d2b7 shop-api: deploy 1.0.43-9f2a1bc to production
b2d9f32 shop-api: deploy 1.0.42-e3b0c44 to production
$ git revert --no-edit f49d2b7 && git push     # Argo CD or Flux rolls 1.0.42 back out

git log --oneline -2

gitversion control
loghistory

git revert --no-edit f49d2b7 && git push

&&run the next command only if this one worked
pushupload commits

Mini mission

Build shop-api's config repository

Create the shop-config repository with a base and a production overlay for shop-api, an Argo CD Application and the Flux equivalent. Validate everything with kustomize and kubeconform, change the image tag the way CI would, and practise a rollback with git revert.

Input: kustomize and kubeconform (single binaries), GitOutput: A config repository whose every commit renders and validates

Reveal one safe solution

$ mkdir -p shop-config/apps/shop-api/{base,production} shop-config/clusters/production && cd shop-config && git init
$ nano apps/shop-api/base/deployment.yaml apps/shop-api/base/service.yaml apps/shop-api/base/kustomization.yaml
$ nano apps/shop-api/production/kustomization.yaml
$ nano clusters/production/argocd-shop-api.yaml clusters/production/flux-shop-api.yaml
$ kustomize build apps/shop-api/production | kubeconform -strict -summary -
Summary: 2 resources found parsing stdin - Valid: 2, Invalid: 0, Errors: 0, Skipped: 0
$ kubeconform -strict -summary -schema-location default \
    -schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
    clusters/production/
Summary: 3 resources found in 2 files - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0
$ git add . && git commit -m "shop-api: production at 1.0.0"
$ (cd apps/shop-api/production && kustomize edit set image ghcr.io/elonerajeev/shop-api:1.0.43-9f2a1bc)
$ git commit -am "shop-api: deploy 1.0.43-9f2a1bc to production"
$ git revert --no-edit HEAD && grep newTag apps/shop-api/production/kustomization.yaml
  newTag: 1.0.0

mkdir -p shop-config/apps/shop-api/{base,production} shop-config/clusters/production && cd shop-config && git init

mkdirmake a directory
-pcreate missing parents, no error if it exists
&&run the next command only if this one worked
cdchange directory
gitversion control
initcreate a repository here

kustomize build apps/shop-api/production | kubeconform -strict -summary -

|pipe: send output to the next command
-the previous directory

git add . && git commit -m "shop-api: production at 1.0.0"

addstage changes
.this folder
commitsave staged changes

git revert --no-edit HEAD && grep newTag apps/shop-api/production/kustomization.yaml

grepfind lines that match a pattern

The 3 resources are the Argo CD Application plus Flux's GitRepository and Kustomization (in practice you'd run one of the two tools; both are here to compare). Every file is in this lesson. With a local cluster (kind), install Argo CD from its getting-started guide and apply clusters/production/argocd-shop-api.yaml to see it sync.

Try it yourself

0 of 6 steps done

Run each task in your own account or terminal, then tick it off.

Write a base Kustomization and a production overlay, and render it with kustomize build.
Change the image tag with kustomize edit set image and read the diff it makes.
Install Argo CD on a local cluster (kind or minikube) and apply the Application from this lesson.
Run kubectl scale deployment shop-api --replicas=5 and watch Argo CD put it back to 2.
Revert the last commit in the config repository and watch the old version roll out.
Validate the manifests with kubeconform before committing them.

Knowledge check

Quick check

Question 1 of 5

What is the main difference between push and pull deployment?

Go deeper

Official reading for this lesson

Or get every quiz answer right and it completes itself.

Lesson 10 · GitOps

GitOps: Git says what runs, the cluster makes it so

Deploy by changing Git, not by pushing to servers: push versus pull, a Kustomize config repo, Argo CD and Flux syncing it to Kubernetes, drift repair, and git revert rollbacks.

40 min read · intermediate · hands-on

By the end of this lesson you'll be able to

0 of 6 complete

Explain push and pull deployment, and what GitOps changes about access and auditing
Lay out a config repository with Kustomize, and change an image tag the way CI does
Define an Argo CD Application that syncs, prunes and self-heals
Define the same deployment in Flux with a GitRepository and a Kustomization
Connect CI to GitOps: build the image, then commit its tag to the config repository
Roll back with git revert, and see drift corrected automatically

Push and pull deploys

	Push (CI applies)	Pull (GitOps)
Who changes the cluster	the pipeline, with cluster credentials	an agent inside the cluster
Credentials outside the cluster	yes, in CI secrets	none: CI only writes to Git
Audit trail	pipeline logs	Git history: who, what, when, approved by whom
Manual changes in the cluster	stay until the next deploy	undone automatically (self-heal)
Rollback	re-run an old pipeline	`git revert`

The OpenGitOps principles sum it up: the desired state is declarative, versioned in Git, pulled automatically, and continuously reconciled.

The config repository

GitOps needs a repository of manifests. A common layout keeps a shared base and one overlay per environment, with Kustomize (built into kubectl):

shop-config/
├── apps/shop-api/
│   ├── base/                  # what every environment shares
│   │   ├── deployment.yaml
│   │   ├── service.yaml
│   │   └── kustomization.yaml
│   └── production/
│       └── kustomization.yaml # production's namespace and image tag
└── clusters/production/
    └── argocd-shop-api.yaml   # tells Argo CD what to sync

The base is an ordinary Deployment and Service:

# apps/shop-api/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop-api
spec:
  replicas: 2
  selector:
    matchLabels: { app: shop-api }
  template:
    metadata:
      labels: { app: shop-api }
    spec:
      containers:
        - name: shop-api
          image: ghcr.io/elonerajeev/shop-api:1.0.0
          ports:
            - containerPort: 3000
          readinessProbe:
            httpGet: { path: /health, port: 3000 }
          resources:
            requests: { cpu: 50m, memory: 64Mi }
            limits: { memory: 128Mi }

# apps/shop-api/base/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: shop-api
spec:
  selector: { app: shop-api }
  ports:
    - port: 80
      targetPort: 3000

# apps/shop-api/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: [deployment.yaml, service.yaml]

The production overlay reuses the base and sets what's specific to production:

# apps/shop-api/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: shop
resources: [../base]
images:
  - name: ghcr.io/elonerajeev/shop-api
    newTag: 1.0.0

A release is one line changing. CI does it with Kustomize:

$ cd apps/shop-api/production
$ kustomize edit set image ghcr.io/elonerajeev/shop-api:1.0.43-9f2a1bc
$ grep -A2 '^images' kustomization.yaml
images:
- name: ghcr.io/elonerajeev/shop-api
  newTag: 1.0.43-9f2a1bc
$ kustomize build . | kubeconform -strict -summary -
Summary: 2 resources found parsing stdin - Valid: 2, Invalid: 0, Errors: 0, Skipped: 0

cd apps/shop-api/production

cdchange directory

grep -A2 '^images' kustomization.yaml

grepfind lines that match a pattern

kustomize build . | kubeconform -strict -summary -

.this folder
|pipe: send output to the next command
-the previous directory

kustomize build renders the final manifests, and kubeconform validates them against the Kubernetes schemas, so a typo fails in CI instead of in the cluster.

Argo CD

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: shop-api
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/elonerajeev/shop-config.git
    targetRevision: main
    path: apps/shop-api/production
  destination:
    server: https://kubernetes.default.svc     # the cluster Argo CD runs in
    namespace: shop
  syncPolicy:
    automated:
      prune: true           # delete what was removed from Git
      selfHeal: true        # undo manual changes in the cluster
    syncOptions:
      - CreateNamespace=true

Setting	Without it
`automated`	Argo CD shows OutOfSync and waits for someone to click Sync
`prune`	a manifest deleted from Git leaves its resource running
`selfHeal`	a `kubectl edit` in production stays until the next commit

The first Application is applied once with kubectl apply -f; after that, Argo CD can manage its own Applications from Git too (the "app of apps" pattern).

Flux

Flux does the same job as a set of small controllers with no central UI, configured entirely with Kubernetes resources: a source to watch, and what to apply from it:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: shop-config
  namespace: flux-system
spec:
  interval: 1m                # check Git every minute
  url: https://github.com/elonerajeev/shop-config
  ref:
    branch: main
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: shop-api
  namespace: flux-system
spec:
  interval: 5m                # re-apply and repair drift every 5 minutes
  sourceRef:
    kind: GitRepository
    name: shop-config
  path: ./apps/shop-api/production
  prune: true
  targetNamespace: shop
  wait: true                  # healthy only when the rollout finishes
  timeout: 2m

	Argo CD	Flux
Shape	one app with a rich web UI	small controllers, CLI-first
Unit	`Application` (and `ApplicationSet` for many)	`GitRepository` + `Kustomization` or `HelmRelease`
Multi-cluster	one Argo CD can manage many clusters	usually Flux in each cluster
Extras	UI, SSO, sync waves, rollouts	image automation that commits new tags for you

CI and GitOps together

CI still builds, tests and publishes the image (lessons 5 and 6). Its last job is just a commit:

update-config:
  runs-on: ubuntu-latest
  environment: production                 # approval before production (lesson 4)
  steps:
    - uses: actions/checkout@v7
      with:
        repository: elonerajeev/shop-config
        token: ${{ secrets.CONFIG_REPO_TOKEN }}   # can write to the config repo only
    - name: Set the new image tag
      working-directory: apps/shop-api/production
      env:
        VERSION: ${{ inputs.version }}
      run: |
        [[ $VERSION =~ ^[0-9]+\.[0-9]+\.[0-9]+-[0-9a-f]{7}$ ]] || { echo "bad version: $VERSION"; exit 1; }
        kustomize edit set image "ghcr.io/elonerajeev/shop-api:$VERSION"
    - name: Commit and push
      env:
        VERSION: ${{ inputs.version }}
      run: |
        git config user.name "shop-api release bot"
        git config user.email "shop-api-bot@users.noreply.github.com"
        git commit -am "shop-api: deploy $VERSION to production"
        git push

Rolling back

$ git log --oneline -2
f49d2b7 shop-api: deploy 1.0.43-9f2a1bc to production
b2d9f32 shop-api: deploy 1.0.42-e3b0c44 to production
$ git revert --no-edit f49d2b7 && git push     # Argo CD or Flux rolls 1.0.42 back out

git log --oneline -2

gitversion control
loghistory

git revert --no-edit f49d2b7 && git push

&&run the next command only if this one worked
pushupload commits

Mini mission

Build shop-api's config repository

Input: kustomize and kubeconform (single binaries), GitOutput: A config repository whose every commit renders and validates

Reveal one safe solution

$ mkdir -p shop-config/apps/shop-api/{base,production} shop-config/clusters/production && cd shop-config && git init
$ nano apps/shop-api/base/deployment.yaml apps/shop-api/base/service.yaml apps/shop-api/base/kustomization.yaml
$ nano apps/shop-api/production/kustomization.yaml
$ nano clusters/production/argocd-shop-api.yaml clusters/production/flux-shop-api.yaml
$ kustomize build apps/shop-api/production | kubeconform -strict -summary -
Summary: 2 resources found parsing stdin - Valid: 2, Invalid: 0, Errors: 0, Skipped: 0
$ kubeconform -strict -summary -schema-location default \
    -schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
    clusters/production/
Summary: 3 resources found in 2 files - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0
$ git add . && git commit -m "shop-api: production at 1.0.0"
$ (cd apps/shop-api/production && kustomize edit set image ghcr.io/elonerajeev/shop-api:1.0.43-9f2a1bc)
$ git commit -am "shop-api: deploy 1.0.43-9f2a1bc to production"
$ git revert --no-edit HEAD && grep newTag apps/shop-api/production/kustomization.yaml
  newTag: 1.0.0

mkdir -p shop-config/apps/shop-api/{base,production} shop-config/clusters/production && cd shop-config && git init

mkdirmake a directory
-pcreate missing parents, no error if it exists
&&run the next command only if this one worked
cdchange directory
gitversion control
initcreate a repository here

kustomize build apps/shop-api/production | kubeconform -strict -summary -

|pipe: send output to the next command
-the previous directory

git add . && git commit -m "shop-api: production at 1.0.0"

addstage changes
.this folder
commitsave staged changes

git revert --no-edit HEAD && grep newTag apps/shop-api/production/kustomization.yaml

grepfind lines that match a pattern

Try it yourself

0 of 6 steps done

Run each task in your own account or terminal, then tick it off.

Write a base Kustomization and a production overlay, and render it with kustomize build.
Change the image tag with kustomize edit set image and read the diff it makes.
Install Argo CD on a local cluster (kind or minikube) and apply the Application from this lesson.
Run kubectl scale deployment shop-api --replicas=5 and watch Argo CD put it back to 2.
Revert the last commit in the config repository and watch the old version roll out.
Validate the manifests with kubeconform before committing them.

Knowledge check

Quick check

Question 1 of 5

What is the main difference between push and pull deployment?

Go deeper

Official reading for this lesson

Or get every quiz answer right and it completes itself.