fluxcd/flux2-multi-tenancy

This project, flux2-multi-tenancy, serves as a starting point for managing multi-tenant clusters using Git and Flux v2. It includes roles for a Platform Admin and Tenant, as well as repository structures for the platform admin and tenant repositories. The Platform Admin manages cluster-wide resources, onboards tenants, and assigns namespaces, service accounts, and role bindings to tenant apps. Tenants have admin access to their assigned namespaces, maintain their Git and app repositories, and manage app deployments and releases using GitRepositories and Kustomizations or HelmRepositories and HelmReleases.

The repository includes a platform admin repository with directories for clusters, infrastructure, and tenants. The tenant repository contains directories for base, staging, and production. The platform admin repository is used to manage cluster-wide resources, while the tenant repository is used for managing tenant-specific applications.

To bootstrap the staging cluster, users must install the Flux CLI, fork the repository, and set up their GitHub username and repo name. They must then verify the staging cluster’s prerequisites, set the context to the staging cluster, and bootstrap Flux using the GitHub token.

New tenants can be onboarded using the Flux CLI to generate the necessary Kubernetes manifests for namespaces, service accounts, and role bindings. The tenant Git repository can also be synced using Flux.

Flux has built-in multi-tenancy lockdown features to enforce tenant isolation at the Control Plane level without the need for external admission controllers like Kyverno. These features include enforcing controllers to block cross-namespace references, denying access to Kustomize remote bases, and setting a default service account for kustomize-controller and helm-controller.

The repository also includes policies for image provenance, Flux Sources, making serviceAccountName mandatory, and reconciliation hierarchy. Additionally, it covers onboarding tenants with private repositories using SSH or token-based authentication and encrypting Kubernetes secrets in Git using Mozilla’s SOPS CLI.

Any changes to the Kubernetes manifests or repository structure should be validated in CI before merging into the main branch and syncing on the cluster. The repository includes GitHub CI workflows for testing the Kubernetes manifests and Kustomize overlays using kubeconform and starting a Kubernetes cluster in CI to test the staging setup using Flux in Kubernetes Kind.