kubernetes-client / csharp

Monitoring and Metrics - kubernetes-client/csharp

Monitoring and Metrics for the Kubernetes client library in C# (kubernetes-client/csharp)

This project provides a .NET client library for the Kubernetes API, which can be used to manage and monitor Kubernetes clusters. Monitoring and metrics are essential to ensure the performance and reliability of the client and the Kubernetes cluster. This document covers the possible options for monitoring the client’s performance using built-in metrics and integrating with external monitoring systems.

Built-in Metrics

The Kubernetes client library for C# includes built-in metrics for monitoring the performance of the client and the Kubernetes cluster. These metrics can be accessed using the PrometheusHandler class, which provides an endpoint for exposing the metrics in a format consumable by Prometheus.

Here’s an example of how to use the PrometheusHandler class to expose the metrics:

using System;
using System.Threading.Tasks;
using k8s;
using k8s.Models;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace MyApp
{
public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}

public IConfiguration Configuration { get; }

public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddKubernetes(Configuration.GetSection("Kubernetes"));
services.AddPrometheusMetrics();
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}

app.UseHttpsRedirection();

app.UseRouting();

app.UseAuthorization();

app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.MapMetrics();
});
}
}
}

In the example above, the AddPrometheusMetrics extension method is used to register the PrometheusHandler class with the DI container. The MapMetrics extension method is then used to map the /metrics endpoint to the PrometheusHandler class.

The built-in metrics include:

  • kubernetes_client_requests_total: A counter for the total number of requests made to the Kubernetes API.
  • kubernetes_client_requests_duration_seconds_sum: A summary for the duration of requests made to the Kubernetes API.
  • kubernetes_client_watch_events_total: A counter for the total number of watch events received from the Kubernetes API.
  • kubernetes_client_watch_errors_total: A counter for the total number of errors while watching resources from the Kubernetes API.

These metrics can be used to monitor the performance of the client and the Kubernetes cluster, such as the number of requests made to the Kubernetes API, the duration of these requests, and the number of watch events and errors.

Integrating with External Monitoring Systems

In addition to the built-in metrics, the Kubernetes client library for C# can be integrated with external monitoring systems, such as Prometheus, Grafana, and Azure Monitor.

Prometheus

Prometheus is an open-source monitoring system that can be used to collect and store metrics from the Kubernetes client library. To integrate Prometheus with the client library, you can use the PrometheusHandler class, as shown in the example above.

Once the PrometheusHandler class is registered with the DI container and mapped to the /metrics endpoint, Prometheus can scrape the metrics by sending an HTTP request to the /metrics endpoint.

Here’s an example of how to configure Prometheus to scrape the metrics from the Kubernetes client library:

scrape_configs:
- job_name: 'myapp'
static_configs:
- targets: ['myapp:80']

In the example above, the myapp job is configured to scrape the metrics from the myapp service on port 80.

Grafana

Grafana is an open-source platform for visualizing and analyzing metrics. To visualize the metrics from the Kubernetes client library in Grafana, you can create a dashboard that queries the metrics from Prometheus.

Here’s an example of how to create a panel in Grafana that queries the kubernetes_client_requests_total metric:

  1. In Grafana, click on the Create button and select Dashboard.
  2. Click on the Add panel button.
  3. Select Graph as the panel type.
  4. In the Metrics tab, enter the following query:
sum(kubernetes_client_requests_total{kubernetes_namespace="<namespace>", kubernetes_pod_name="<pod>"})

Replace <namespace> and <pod> with the namespace and pod name of the Kubernetes client library.

  1. Click on the Apply button.

The panel will now display the kubernetes_client_requests_total metric for the Kubernetes client library.

Azure Monitor

Azure Monitor is a monitoring service that can be used to collect and analyze metrics from the Kubernetes client library. To integrate Azure Monitor with the client library, you can use the Azure Monitor .NET agent, which can be installed as a sidecar container in the same pod as the Kubernetes client library.

Here’s an example of how to configure the Azure Monitor .NET agent to collect metrics from the Kubernetes client library:

  1. Install the Azure Monitor .NET agent as a sidecar container in the same pod as the Kubernetes client library.
  2. Configure the Azure Monitor .NET agent to collect metrics from the Kubernetes client library by adding the following configuration to the azmona.json file:
{
"metrics": [
{
"name": "kubernetes_client_requests_total",
"source": "MyApp.K8sClientMetrics",
"sink": "AzureMonitor"
}
]
}

In the example above, the kubernetes_client_requests_total metric is configured to be collected from the MyApp.K8sClientMetrics class and sent to Azure Monitor.

  1. Restart the Azure Monitor .NET agent to apply the configuration changes.

The metrics from the Kubernetes client library will now be collected and sent to Azure Monitor.

Conclusion

Monitoring and metrics are essential for ensuring the performance and reliability of the Kubernetes client library and the Kubernetes cluster. The Kubernetes client library for C# provides built-in metrics that can be used to monitor the performance of the client and the Kubernetes cluster. Additionally, the client library can be integrated with external monitoring systems, such as Prometheus, Grafana, and Azure Monitor, to visualize and analyze the metrics. By using these monitoring and metrics tools, you can ensure that your Kubernetes client library and cluster are performing optimally and identify any issues before they become critical.

Sources: