kubernetes / client-go

Security Best Practices for Kubernetes Client-Go

Motivation: Securing Kubernetes clusters and applications is paramount to protect sensitive data and prevent unauthorized access. The client-go library is a powerful tool for interacting with Kubernetes clusters, but it’s crucial to implement security best practices when using it.

What’s included:

  • Clientset for accessing Kubernetes API (kubernetes package).
  • API discovery (discovery package).
  • Dynamic client for generic operations on Kubernetes API objects (dynamic package).
  • Authentication plugins for obtaining credentials from external sources (plugin/pkg/client/auth packages).
  • Connection setup and authentication (transport package).
  • Tools for writing controllers (tools/cache package).

Versioning:

  • Refer to the Kubernetes main repository for the latest version information.

Compatibility:

  • Your code with client-go: The client-go library is compatible with the Kubernetes versions it targets. Consult the official Kubernetes documentation for compatibility information.
  • client-go with Kubernetes clusters: Client-go is compatible with Kubernetes clusters running the same or a newer version of the API. You can use the discovery package to discover the supported API versions of a cluster.

Compatibility Matrix:

client-go version Kubernetes version
1.19 1.19, 1.20, 1.21, 1.22, 1.23, 1.24
1.20 1.20, 1.21, 1.22, 1.23, 1.24
1.21 1.21, 1.22, 1.23, 1.24
1.22 1.22, 1.23, 1.24
1.23 1.23, 1.24
1.24 1.24

Why do the 1.4 and 1.5 branch contain top-level folders named after the version?

The 1.4 and 1.5 branches contain top-level folders named after the version to indicate the specific version of the Kubernetes API they support.

Kubernetes tags:

  • The client-go repository uses tags to denote specific versions. You can use these tags to access a specific release.

How to get it:

  • Use git clone to retrieve the repository.
  • Use go get to install the library.

How to use it:

  • Refer to the examples in the examples directory for detailed usage instructions.
  • Consult the official Kubernetes documentation for general usage information.

Dependency Management:

  • Use go mod for dependency management.
  • Include the client-go library as a dependency in your project’s go.mod file.

Contributing code:

  • Submit pull requests against the client packages in the Kubernetes main repository. Changes in the staging area will be published to the client-go repository daily.

Examples:

  • Work queues: Utilize the rate-limited workqueue and informer framework to create a hotloop-free controller [informer]: https://godoc.org/k8s.io/client-go/tools/cache#NewInformer.
  • Custom Resource Definition (CRD): Register a custom resource type, create/update/query it, and build a controller to manage cluster state based on changes.
  • Leader election: Implement HA controllers using the leader election package.
  • Managing resources with API: Create, get, update, and delete resources like Deployments.

Client Configuration:

  • Typically loaded from kubeconfig files containing server and credential configuration.
  • External plugins for obtaining credentials are available but not loaded by default.
  • To enable plugins, import them in your main package:
import _ "k8s.io/client-go/plugin/pkg/client/auth"
  • Load specific plugins:
import _ "k8s.io/client-go/plugin/pkg/client/auth/azure"
          import _ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
          import _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"

Dynamic Package:

  • Defers data bindings until runtime, offering flexibility in handling API changes.
  • May lack type validation benefits until runtime, potentially impacting applications requiring strong data type checks.

In-cluster Client Configuration:

SecurityContextApplyConfiguration:

  • Provides methods to configure security context settings.
  • Methods like WithPrivileged, WithAllowPrivilegeEscalation, WithSELinuxOptions, WithSeccompProfile, WithCapabilities, WithRunAsUser, WithReadOnlyRootFilesystem, and WithRunAsNonRoot allow setting specific security context options.

WindowsSecurityContextOptionsApplyConfiguration:

  • Constructs a declarative configuration for Windows security context options.

Auth Configuration:

  • The authcfg.Info struct defines the format for storing Kubernetes API authorization configuration.
  • This format allows for portability across different authentication styles.

ExecCredentialStatus:

  • Holds credentials for transport, including token, client certificate data, and client key data.
  • Sensitive data is transmitted in-memory between client and exec plugin process for security.
  • Exec plugin itself should be protected via file permissions.

Top-Level Directory Explanations

applyconfigurations/ - This directory contains examples and tests for applying Kubernetes configurations using the client-go library.

discovery/ - This directory contains code related to service discovery in Kubernetes.

dynamic/ - This directory contains code for working with dynamic resources in the Kubernetes API.

examples/ - This directory contains example usage of the client-go library.

features/ - This directory contains experimental features that are not yet stable.

informers/ - This directory contains code for caching Kubernetes resources using informers.

kubernetes/ - This directory contains the main package for the client-go library, which provides types and functions for interacting with the Kubernetes API.

listers/ - This directory contains interfaces and implementations for listing Kubernetes resources.

metadata/ - This directory contains code related to metadata in Kubernetes.

pkg/ - This directory contains compiled Go packages for the client-go library.

plugin/ - This directory contains code for loading plugins for the client-go library.

rest/ - This directory contains code for working with the REST API in the client-go library.

scale/ - This directory contains code for working with scale and autoscaling in Kubernetes.

tools/ - This directory contains various tools for working with the client-go library.

util/ - This directory contains utility functions for the client-go library.