Metrics and Observability in Jaeger
Jaeger, an open-source distributed tracing system, provides various metrics libraries for monitoring and troubleshooting distributed systems. These libraries include expvar, Prometheus, and go-kit, each serving a unique purpose.
expvar
The expvar
library is a Go standard mechanism for exposing process level statistics. It provides a simple HTTP server that serves a directory of variables that can be incremented, decremented, or read. Jaeger utilizes expvar
to expose metrics in a Prometheus-compatible format.
Example usage:
import "expvar"
// create a new int variable
myVar := expvar.NewInt("my_variable")
// increment the variable
myVar.Add(1)
Prometheus
Prometheus is an open-source systems monitoring and alerting toolkit. Jaeger integrates with Prometheus to expose metrics in a time-series format. This integration allows for powerful querying and visualization of metrics.
Example usage:
import "github.com/prometheus/client_golang/prometheus"
// create a new counter
myCounter := prometheus.NewCounter(prometheus.CounterOpts{
Name: "my_counter",
Help: "A simple counter",
})
// register the counter with the Prometheus registry
prometheus.MustRegister(myCounter)
// increment the counter
myCounter.Inc()
go-kit
go-kit is a collection of libraries for building microservices in Go. Jaeger includes a go-kit
metrics library that provides a simple and consistent interface for creating and registering metrics.
Example usage:
import "github.com/go-kit/kit/metrics"
// create a new counter
myCounter := metrics.NewCounter(metrics.WithName("my_counter"))
// increment the counter
myCounter.With("method", "my_method").Add(1)
Metrics can be used to monitor and troubleshoot distributed systems by providing insights into the behavior and performance of these systems. By utilizing the metrics libraries provided by Jaeger, users can build a robust monitoring infrastructure for their Jaeger installation.
Sources:
- Introduction to open source observability on Kubernetes | Opensource.com: https://opensource.com/article/19/10/open-source-observability-kubernetes
- Build a monitoring infrastructure for your Jaeger installation | Red Hat Developer: https://developers.redhat.com/blog/2019/08/28/build-a-monitoring-infrastructure-for-your-jaeger-installation
- Monitoring Jaeger — Jaeger documentation: https://www.jaegertracing.io/docs/next-release/monitoring
- How to manage high cardinality metrics in Prometheus and Kubernetes | Grafana Labs: https://grafana.com/blog/2022/10/20/how-to-manage-high-cardinality-metrics-in-prometheus-and-kubernetes
- Service Performance Monitoring (SPM) — Jaeger documentation: https://www.jaegertracing.io/docs/next-release/spm
- Boundary Metrics | Boundary | HashiCorp Developer: https://developer.hashicorp.com/boundary/docs/operations/metrics
- Automated Observability | KubeVela: https://kubevela.io/docs/platform-engineers/operations/observability
- HTTP Requests Latency | Kyverno: https://kyverno.io/docs/monitoring/http-requests-latency
- OpenTelemetry | Kyverno: https://kyverno.io/docs/monitoring/opentelemetry
- cert-manager packaged by Bitnami for Kubernetes: https://docs.bitnami.com/kubernetes/infrastructure/cert-manager
- Intro to Observability with Prometheus and beyond | Grafana Labs: https://grafana.com/go/webinar/intro-to-observability-with-prometheus