Infrastructure as Code (IaC)

This repository provides a Flux-based infrastructure as code (IaC) solution for managing multiple Kubernetes clusters. The goal is to automate cluster setup and management while minimizing duplicated declarations. This document outlines the structure and organization of the IaC codebase.

Repository Structure:

The repository is structured into three top-level directories:

• apps: Contains Helm releases with custom configurations per cluster.
  • apps/base/: Contains namespaces and Helm release definitions for common configurations.
  • apps/production/: Contains production-specific Helm release values.
  • apps/staging/: Contains staging-specific Helm release values.
• infrastructure: Contains shared infrastructure tools and configurations.
  • infrastructure/controllers/: Contains namespaces and Helm release definitions for Kubernetes controllers.
  • infrastructure/configs/: Contains Kubernetes custom resources, such as cert issuers and network policies.
• clusters: Contains Flux configurations per cluster.
  • clusters/production/: Contains the Flux configurations for the production cluster.
  • clusters/staging/: Contains the Flux configurations for the staging cluster.

Infrastructure Configuration:

The infrastructure directory contains common infrastructure components, managed by Flux using Helm and Kustomize:

• infrastructure/controllers/: This directory contains namespaces and Helm release definitions for Kubernetes controllers like cert-manager and ingress-nginx.
  • infrastructure/controllers/cert-manager.yaml: Defines a HelmRelease for the cert-manager chart, including the repository and version.
  • infrastructure/controllers/ingress-nginx.yaml: Defines a HelmRelease for the ingress-nginx chart, including the repository and version.
• infrastructure/configs/: This directory contains Kubernetes custom resources, such as the Let’s Encrypt issuer:
  • infrastructure/configs/cluster-issuers.yaml: Defines a ClusterIssuer for Let’s Encrypt, using the staging or production API depending on the cluster.

Cluster Configurations:

Each cluster has its dedicated configuration directory under clusters:

• clusters/production/: Contains the Flux configurations for the production cluster.
  • clusters/production/infrastructure.yaml: Defines two Kustomizations:
    • infra-controllers: Installs the controllers from the infrastructure/controllers directory.
    • infra-configs: Applies the configuration from infrastructure/configs, including the Let’s Encrypt issuer with the production API.
• clusters/staging/: Contains the Flux configurations for the staging cluster.
  • clusters/staging/infrastructure.yaml: Similar to clusters/production/infrastructure.yaml, but using the staging API for the Let’s Encrypt issuer.

Application Configurations:

The apps directory contains the applications managed by Flux using Helm and Kustomize.

• apps/base/: Contains the base configuration for the podinfo application.
  • apps/base/podinfo/namespace.yaml: Defines the podinfo namespace.
  • apps/base/podinfo/repository.yaml: Defines a HelmRepository for the podinfo chart.
  • apps/base/podinfo/release.yaml: Defines a HelmRelease for the podinfo chart, specifying the repository, chart name, and version range.
• apps/production/: Contains production-specific customizations for podinfo.
  • apps/production/podinfo-patch.yaml: Contains patches specific to the production cluster.
• apps/staging/: Contains staging-specific customizations for podinfo.
  • apps/staging/podinfo-patch.yaml: Contains patches specific to the staging cluster.

Deployment:

Flux deploys the infrastructure and applications to each cluster based on the configurations defined in the clusters directory. The deployment process involves the following steps:

• Bootstrap: The flux bootstrap command initializes Flux on a new cluster, configuring the Git repository as the source of truth for the IaC.
• Synchronization: Flux monitors the Git repository for changes and automatically updates the cluster configuration accordingly.
• Dependencies: The dependsOn field in the Kustomizations ensures that infrastructure components are deployed before applications.

Testing:

The repository includes continuous integration (CI) workflows for validating the IaC:

• Test workflow: Uses kubeconform to validate the Kubernetes manifests and Kustomize overlays.
• E2E workflow: Deploys the entire stack to a Kubernetes cluster in CI, running Flux and testing the setup.

Notes:

• This approach uses Flux and Kustomize to manage the cluster configuration, ensuring consistency and reproducibility.
• The interval fields in the Kubernetes custom resources determine the frequency at which Flux pulls updates from the Git repository.
• The prune field in the Kustomizations enables Flux to automatically remove outdated resources.

Further Information:

For more information on Flux and Kustomize, refer to the following resources:

• Flux
• Kustomize

Code Files:

• README.md
• .sourceignore
• apps/base/podinfo/namespace.yaml
• infrastructure/controllers/ingress-nginx.yaml
• clusters/production/infrastructure.yaml
• clusters/staging/infrastructure.yaml
• apps/base/podinfo/repository.yaml
• infrastructure/controllers/kustomization.yaml
• clusters/production/apps.yaml
• infrastructure/configs/kustomization.yaml
• clusters/staging/apps.yaml
• clusters/production/flux-system/kustomization.yaml
• clusters/staging/flux-system/kustomization.yaml
• scripts/validate.sh
• apps/production/podinfo-values.yaml
• apps/staging/podinfo-values.yaml

Top-Level Directory Explanations

apps/ - This directory contains the application definitions and configurations. It includes base, production, staging directories.

apps/base/ - This directory contains the base application configurations. It includes podinfo directory.

apps/production/ - This directory contains the production application configurations.

apps/staging/ - This directory contains the staging application configurations.

clusters/ - This directory contains the Kubernetes cluster configurations. It includes production and staging directories.

clusters/production/ - This directory contains the production Kubernetes cluster configurations. It includes flux-system directory.

clusters/staging/ - This directory contains the staging Kubernetes cluster configurations. It includes flux-system directory.

infrastructure/ - This directory contains the infrastructure configurations and scripts.

infrastructure/configs/ - This directory contains the configuration files for the infrastructure.

infrastructure/controllers/ - This directory contains the Kubernetes controllers for managing the infrastructure.

scripts/ - This directory contains the scripts used for automating tasks.