Controllers
Motivation
This section outlines the core concepts and purpose of Controllers in Kubernetes, with a focus on Deployment, ReplicaSet, StatefulSet, and DaemonSet. These controllers are responsible for managing workloads and ensuring that the desired state is achieved and maintained.
Controller-manager
This repository serves as a central location for common code shared across controller managers, primarily the Kube-Controller-Manager and Cloud-Controller-Manager. This shared code promotes consistency and best practices in controller design and implementation. [Source: staging/src/k8s.io/controller-manager/README.md]
Compatibility
It’s crucial to note that there are no compatibility guarantees for this repository. This repository is directly tied to the Kubernetes project and its branches are aligned with Kubernetes releases. Therefore, compatibility is maintained with the corresponding Kubernetes release. Future efforts will focus on enhancing compatibility and making this codebase more readily accessible. [Source: staging/src/k8s.io/controller-manager/README.md]
Origins
The code within this repository stems from the common functionalities found in both Kube-Controller-Manager and Cloud-Controller-Manager. The goal is to encapsulate best practices for building controller managers, ensuring consistency and efficiency. Legacy aspects of these controller managers will be addressed before being integrated into this repository. Changes to the code are made in the k8s.io/kubernetes/staging/src/k8s.io/controller-manager repository, merged into k8s.io/kubernetes, and later synchronized here. [Source: staging/src/k8s.io/controller-manager/README.md]
Prohibited Actions
- Direct Modification of
pkgDirectory: Modifying files directly under thepkgdirectory is discouraged. These files are managed byk8s.io/kubernetes/staging/src/k8s.io/controller-manager. [Source: staging/src/k8s.io/controller-manager/README.md] - Compatibility Expectations: Due to its close relationship with Kubernetes and ongoing development efforts, the repository undergoes frequent changes, making compatibility guarantees unreliable. [Source: staging/src/k8s.io/controller-manager/README.md]
Controller Expectations
Controllers use expectations to inform the controller manager about anticipated events. These expectations specify the number of expected events (e.g., additions or deletions) within a specific timeframe.
- ControlleeExpectation: This structure tracks the creation and deletion of controlled resources using atomic counters.
- ControllerExpectationsStore: A store that combines a TTLStore with a ControlleeExpectation per controller.
Key points:
- Once expectations are set, they can only be reduced.
- A controller remains unsynchronized until its expectations are met or expire.
- Controllers that don’t set expectations will be woken up for every matching controllee.
Code snippets demonstrating these concepts:
// ControllerExpectations is a cache mapping controllers to what they expect to see before being woken up for a sync.
type ControllerExpectations struct {
cache.Store
}
// Expectations are a way for controllers to tell the controller manager what they expect. eg:
// ControllerExpectations: {
// controller1: expects 2 adds in 2 minutes
// controller2: expects 2 dels in 2 minutes
// controller3: expects -1 adds in 2 minutes => controller3's expectations have already been met
// }
//
// Implementation:
// ControlleeExpectation = pair of atomic counters to track controllee's creation/deletion
// ControllerExpectationsStore = TTLStore + a ControlleeExpectation per controller
//
// * Once set expectations can only be lowered
// * A controller isn't synced till its expectations are either fulfilled, or expire
// * Controllers that don't set expectations will get woken up for every matching controllee
// ExpKeyFunc to parse out the key from a ControlleeExpectation
ExpKeyFunc
Controller References
Controller references provide a mechanism to define the relationship between a controller and its controlled resources. This enables controllers to manage the lifecycle of their controlled resources.
Code Snippet demonstrating these concepts:
// GetControllerOf returns a pointer to a copy of the controllerRef if controllee has a controller
func GetControllerOf(controllee Object) *OwnerReference {
ref := GetControllerOfNoCopy(controllee)
if ref == nil {
return nil
}
cp := *ref
cp.Controller = ptr.To(*ref.Controller)
if ref.BlockOwnerDeletion != nil {
cp.BlockOwnerDeletion = ptr.To(*ref.BlockOwnerDeletion)
}
return &cp
}
Client-go and Custom Controllers
Client-go provides a set of tools that simplify the development of custom controllers. These tools facilitate communication with the Kubernetes API and provide mechanisms for managing resources.
Key Client-go Components:
- Reflector: Watches the Kubernetes API for a specific resource type, handling events like creation, deletion, and updates.
- Informer: Processes objects from a queue, handling events received by the Reflector, and triggering actions within the custom controller.
- Indexer: Provides indexing functionality to enable efficient retrieval of objects based on specific criteria, such as labels.
Custom Controller Components:
- Informer Reference: A reference to the Informer instance responsible for managing the custom resource.
- Indexer Reference: A reference to the Indexer instance used to retrieve custom resources.
- Resource Event Handlers: Callback functions triggered by the Informer to deliver objects to the controller.
- Work Queue: A queue used by the controller to decouple the delivery of objects from their processing.
- Process Item: A function that processes items from the work queue.
Code snippets demonstrating these concepts:
// ControllerRevisionControllerRefManager is used to manage controllerRef of ControllerRevisions.
// Three methods are defined on this object 1: Classify 2: AdoptControllerRevision and
// 3: ReleaseControllerRevision which are used to classify the ControllerRevisions into appropriate
// categories and accordingly adopt or release them. See comments on these functions
// for more details.
type ControllerRevisionControllerRefManager struct {
BaseControllerRefManager
controllerKind schema.GroupVersionKind
crControl ControllerRevisionControlInterface
}
Controller Names and Aliases
Controller names are used to identify and manage controllers. Controller aliases provide alternative names for controllers, facilitating easier referencing.
func getAllControllers() []string {
return []string{
"red-controller",
"green-controller",
"blue-controller",
}
}
func getControllerAliases() map[string]string {
return map[string]string{
"crimson-controller": "red-controller",
"pink-controller": "red-controller",
"ultramarine-controller": "blue-controller",
}
}
ControllerRevisions
ControllerRevisions are objects used to manage the state of controllers, providing a mechanism to rollback to specific configurations.
// ControllerRevisionNamespaceLister helps list and get ControllerRevisions.
// All objects returned here must be treated as read-only.
type ControllerRevisionNamespaceLister interface {
// List lists all ControllerRevisions in the indexer for a given namespace.
// Objects returned here must be treated as read-only.
List(selector labels.Selector) (ret []*v1beta2.ControllerRevision, err error)
// Get retrieves the ControllerRevision from the indexer for a given namespace and name.
// Objects returned here must be treated as read-only.
Get(name string) (*v1beta2.ControllerRevision, error)
ControllerRevisionNamespaceListerExpansion
}
DaemonSet
DaemonSet controllers are responsible for ensuring that a specific pod runs on every node in the cluster.
controllerKind
Deployment
Deployment controllers manage the deployment of applications, ensuring that the desired number of replicas are running and handling updates gracefully.
ReplicaSet
ReplicaSet controllers provide a foundational building block for managing deployments. They ensure that the specified number of pods are running with consistent configurations.
StatefulSet
StatefulSet controllers manage stateful applications, ensuring that pods maintain persistent storage and unique identities throughout their lifecycle.
Conclusion
This documentation provides a foundation for understanding controllers in Kubernetes. By leveraging these concepts, developers can effectively build and manage their applications within the Kubernetes ecosystem.
Top-Level Directory Explanations
build - This directory contains the build scripts and configurations for Kubernetes components. It includes subdirectories like build-image, lib, pause, and server-image that build different parts of Kubernetes, such as images, libraries, and server components.
cmd - This directory contains the command-line interface (CLI) tools for Kubernetes, such as clicheck, cloud-controller-manager, dependencycheck, dependencyverifier, and kubectl.
hack - This directory contains development scripts and tools for the Kubernetes project. It includes subdirectories like boilerplate, conformance, e2e-internal, gen-swagger-doc, jenkins, lib, make-rules, testdata, and tools.
pkg - This directory contains the Go packages for Kubernetes components. It includes subdirectories like api, apis, auth, capabilities, client, cluster, controller, controlplane, credentialprovider, features, fieldpath, generated, kubeapiserver, kubectl, kubemark, printers, probe, proxy, quota, registry, routes, scheduler, security, serviceaccount, util, volume, and windows. These packages define and implement various Kubernetes features and components.
plugin - This directory contains the plugins for Kubernetes, such as admission plugins and auth plugins. It includes subdirectories like pkg and auth.
test - This directory contains the test scripts and configurations for Kubernetes components. It includes subdirectories like cmd, conformance, e2e, fixtures, fuzz, images, kubemark, e2e_kubeadm, e2e_node, and utils.
