This documentation provides a step-by-step guide to configuring the development environment for FluxCD’s multi-tenancy setup. The examples given are specifically tailored for expert developers familiar with the FluxCD and Kubernetes ecosystem.

1. Setting Up the Structure

Begin by creating the necessary directory structure for your tenant. This will include namespaces, service accounts, role bindings, and repositories.

mkdir -p ./tenants/base/dev-team

2. Creating Tenant RBAC

Next, generate the namespace, service account, and role binding for the tenant. This can be done using the Flux CLI as shown below:

flux create tenant dev-team --with-namespace=apps \
--export > ./tenants/base/dev-team/rbac.yaml

The content of the rbac.yaml file will look like this:

---
apiVersion: v1
kind: Namespace
metadata:
  labels:
    toolkit.fluxcd.io/tenant: dev-team
  name: apps

---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    toolkit.fluxcd.io/tenant: dev-team
  name: dev-team
  namespace: apps

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    toolkit.fluxcd.io/tenant: dev-team
  name: gotk-reconciler
  namespace: apps
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: User
  name: gotk:apps:reconciler
- kind: ServiceAccount
  name: dev-team
  namespace: apps

3. Creating Sync Manifests for Git Repository

To link the tenant with its respective Git repository, create the sync manifests referencing the appropriate authentication secret:

flux create source git dev-team \
--namespace=apps \
--url=https://github.com/<your-repo> \
--branch=main \
--secret-ref=dev-team-auth \
--export > ./tenants/base/dev-team/sync.yaml

4. Kustomization Setup

The following commands will set up the Kustomization for the tenant. The Kustomization will reference the source defined in the previous step.

First, create the Kustomization YAML file:

flux create kustomization dev-team \
--namespace=apps \
--service-account=dev-team \
--source=GitRepository/dev-team \
--path="./" \
--export >> ./tenants/base/dev-team/sync.yaml

Next, create the base kustomization.yaml file in the tenant directory:

cd ./tenants/base/dev-team/ && kustomize create --autodetect

5. Encrypted Secrets Management with SOPS

Ensure your sensitive data, like Git authentication secrets, is encrypted. For instance, encrypt the dev-team-auth secret:

sops --encrypt \
--pgp=1F3D1CED2F865F5E59CA564553241F147E7C5FA4 \
--encrypted-regex '^(data|stringData)$' \
--in-place ./tenants/base/dev-team/auth.yaml

6. Configuring Flux to Decrypt Secrets

In your Kustomization, configure Flux to use sops for decrypting secrets. The flow would typically look like this:

flux create kustomization tenants \
--depends-on=kyverno-policies \
--source=flux-system \
--path="./tenants/staging" \
--prune=true \
--interval=5m \
--validation=client \
--decryption-provider=sops \
--decryption-secret=sops-gpg \
--export > ./clusters/staging/tenants.yaml

This configuration enables the Flux instance running in the staging cluster to manage decrypted secrets appropriately.

7. Kustomization for Production and Staging

For production and staging environments, you may want to set up individual Kustomizations that allow for different configurations and values:

Staging Kustomization

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: apps
resources:
  - ../base/dev-team
patches:
  - path: dev-team-patch.yaml

Production Kustomization

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: apps
resources:
  - ../base/dev-team
patches:
  - path: dev-team-patch.yaml

This sets a foundation for both environments. It allows the control of different configuration values while managing the same underlying resources.

References

This documentation references the file structure and contents drawn from the provided sources. For in-depth review, please refer to the FluxCD documentation and your local repository structure.