This documentation outlines the CI/CD workflow for the fluxcd/flux2-multi-tenancy project. The details herein provide a comprehensive step-by-step guide to ensure effective implementation of continuous integration and continuous deployment using FluxCD.

Prerequisites

Before proceeding with the CI/CD setup, ensure that the following prerequisites are satisfied:

1. Install the Flux CLI:
   Follow the official installation instructions for the Flux CLI.

2. Fork the repository:
   Fork the fluxcd/flux2-multi-tenancy repository to your personal GitHub account.

3. Set environment variables:
   Export your GitHub username and repository name:

   export GITHUB_USER=your_github_username
   export GITHUB_REPO=your_repo_name
   

4. Validate the staging cluster:
   Ensure that your staging cluster meets the requirements:

   flux check --pre
   

Bootstrapping Flux

Once the prerequisites are validated, bootstrap Flux with the following command:

flux bootstrap github \
--context=your-staging-context \
--owner=${GITHUB_USER} \
--repository=${GITHUB_REPO} \
--branch=main \
--personal \
--path=clusters/staging

During this process, the Flux CLI will require your GITHUB_TOKEN. This token is securely used by Flux to configure the repository and set up a deploy key for pulling changes in the cluster.

Setting Up CI Workflows

This project includes two CI workflows that are critical for validating changes before they are merged into the main branch:

1. Testing Workflow:
   This workflow validates the Kubernetes manifests and Kustomize overlays using kubeconform. The validation script can be found at scripts/validate.sh.

   To execute the validation script, run:

   bash scripts/validate.sh
   

   The key part of the script is shown below:

   echo "INFO - Validating kustomize overlays"
   find . -type f -name $kustomize_config -print0 | while IFS= read -r -d $'\0' file;
   do
       echo "INFO - Validating kustomization ${file/%$kustomize_config}"
       kustomize build "${file/%$kustomize_config}" "${kustomize_flags[@]}" | \
       kubeconform "${kubeconform_flags[@]}" "${kubeconform_config[@]}"
       if [[ ${PIPESTATUS[0]} != 0 ]]; then
           exit 1
       fi
   done
   

2. End-to-End (e2e) Workflow:
   This workflow starts a Kubernetes cluster in CI and tests the staging setup by running Flux in Kubernetes’ Kind.

Tenants Structure

Each tenant is represented with its own Git repository, which includes directories for application manifests:

• base: Contains the HelmRepository and HelmRelease manifests.
• staging: Contains HelmRelease Kustomize patches for deploying pre-releases.

Example Tenant Configuration

Example configuration in a tenant repository might look like this:

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: dev-team
  namespace: apps
spec:
  interval: 1m
  url: https://github.com/fluxcd/flux2-multi-tenancy
  ref:
    branch: dev-team
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: dev-team
  namespace: apps
spec:
  serviceAccountName: dev-team
  interval: 5m
  sourceRef:
    kind: GitRepository
    name: dev-team
  prune: true

CI/CD Integration Steps

1. Create Git Credentials:
   Generate a Kubernetes secret to store Git credentials securely:

   flux -n apps create secret git dev-team-auth \
   --url=https://github.com/ \
   --username=$GITHUB_USERNAME \
   --password=$GITHUB_TOKEN \
   --export > ./tenants/base/dev-team/auth.yaml
   

2. Monitor Kustomizations:
   Monitor the reconciliation status of the Kustomizations:

   flux get kustomizations --watch
   

Conclusion

This documentation provides steps to implement CI/CD workflows using FluxCD for multi-tenant Kubernetes environments. Any changes to Kubernetes manifests or repository structures should be validated in CI before merging pull requests.

Next Steps

If CI/CD workflows are not yet set up, consider implementing the above strategies to enhance the development and deployment processes within the project.

Sources:

• README.md
• infrastructure/kyverno-policies/verify-git-repositories.yaml
• scripts/validate.sh
• tenants/base/dev-team/sync.yaml
• clusters/staging/infrastructure.yaml