Key Concepts

This repository uses Git, GitHub, and Flux to manage two clusters, staging and production. The code is structured into three main directories:

• apps: This directory contains Helm releases with custom configurations per cluster. The directory is structured as follows:

  • base: Contains namespaces and Helm release definitions for common resources.
  • production: Contains the production Helm release values.
  • staging: Contains the staging Helm release values.

• infrastructure: This directory contains common infrastructure tools such as ingress-nginx and cert-manager. The directory is structured as follows:

  • configs: Contains Kubernetes custom resources such as cert issuers and network policies.
  • controllers: Contains namespaces and Helm release definitions for Kubernetes controllers.

• clusters: This directory contains the Flux configuration per cluster. The directory is structured as follows:

  • production: Contains the Flux configuration for the production cluster.
  • staging: Contains the Flux configuration for the staging cluster.

Configuration and Deployment

The project uses Kustomize overlays to manage the configuration for each cluster. For example, the clusters/staging/apps.yaml file defines a Kustomization object that uses the path field to configure Flux to sync the staging Kustomize overlay.

The dependsOn field is used to tell Flux to create the infrastructure items before deploying the apps.

apiVersion: kustomize.toolkit.fluxcd.io/v1
          kind: Kustomization
          metadata:
          name: apps
          namespace: flux-system
          spec:
          interval: 10m0s
          dependsOn:
          - name: infra-configs
          sourceRef:
          kind: GitRepository
          name: flux-system
          path: ./apps/staging
          prune: true
          wait: true
          

The apps/base/podinfo directory contains a Flux HelmRelease with common values for both clusters:

apiVersion: helm.toolkit.fluxcd.io/v2
          kind: HelmRelease
          metadata:
          name: podinfo
          namespace: podinfo
          spec:
          releaseName: podinfo
          chart:
          spec:
          chart: podinfo
          sourceRef:
          

The apps/staging/kustomization.yaml file defines a Kustomization object that applies a patch to the HelmRelease.

apiVersion: kustomize.config.k8s.io/v1beta1
          kind: Kustomization
          namespace: podinfo
          resources:
            - ../base/podinfo
          patches:
            - path: podinfo-values.yaml
              target:
                kind: HelmRelease
          

CI/CD

The repository contains GitHub CI workflows that validate the Kubernetes manifests and Kustomize overlays and test the staging setup by running Flux in Kubernetes Kind.

Adding a New Cluster

To add a new cluster, follow these steps:

1. Clone the repository locally.
2. Create a directory inside clusters with your cluster name.
3. Copy the sync manifests from staging.
4. Push the changes to the main branch.
5. Set the kubectl context and path to your new cluster.
6. Bootstrap Flux.

Automatic Upgrades

Flux automatically upgrades Helm releases to the latest stable chart version based on semver ranges.

apiVersion: helm.toolkit.fluxcd.io/v2
          kind: HelmRelease
          metadata:
          name: podinfo
          namespace: podinfo
          spec:
          chart:
          spec:
          version: ">=1.0.0"
          values:
          ingress:
          hosts:
          - host: podinfo.production
          

Bootstrapping Flux

To bootstrap Flux, use the flux bootstrap github command. This command commits the manifests for the Flux components in clusters/staging/flux-system and creates a deploy key with read-only access on GitHub so that Flux can pull changes inside the cluster.

flux bootstrap github \
          --context=staging \
          --owner=${GITHUB_USER} \
          --repository=${GITHUB_REPO} \
          --branch=main \
          --personal \
          --path=clusters/staging
          

Best Practices

The repository follows these best practices:

• Use Git for version control.
• Use GitHub for collaboration and CI/CD.
• Use Flux for automated deployments and upgrades.
• Use Kustomize for managing configuration per cluster.
• Validate changes in CI before merging.
• Test changes in a staging environment before deploying to production.

Top-Level Directory Explanations

apps/ - This directory contains the application definitions and configurations. It includes base, production, staging directories.

apps/base/ - This directory contains the base application configurations. It includes podinfo directory.

apps/production/ - This directory contains the production application configurations.

apps/staging/ - This directory contains the staging application configurations.

clusters/ - This directory contains the Kubernetes cluster configurations. It includes production and staging directories.

clusters/production/ - This directory contains the production Kubernetes cluster configurations. It includes flux-system directory.

clusters/staging/ - This directory contains the staging Kubernetes cluster configurations. It includes flux-system directory.

infrastructure/ - This directory contains the infrastructure configurations and scripts.

infrastructure/configs/ - This directory contains the configuration files for the infrastructure.

infrastructure/controllers/ - This directory contains the Kubernetes controllers for managing the infrastructure.

scripts/ - This directory contains the scripts used for automating tasks.