Internal Logging in Jaeger-lib
Jaeger-lib, a distributed tracing system, has built-in logging mechanisms that can be used to gather useful information for debugging. This document will explore the possible options for internal logging in Jaeger-lib using the provided documentation and code snippets.
- Setting the Log Level
The log level can be adjusted in Jaeger-lib using the --log-level
command line switch. The default level is info
, but it can be changed to debug
for more detailed logging. For example, in a Jaeger Operator deployment, the log level can be set as follows:
args : [ "start" , "--log-level=debug" ]
Or, in an environment variable:
env :
- name : LOG-LEVEL
value : debug
- Using the Logging Reporter
Most Jaeger SDKs are able to log the spans that are being reported to the logging facility provided by the instrumented application. This can be done by setting the environment variable JAEGER_REPORTER_LOG_SPANS
to true
. In some languages, such as Go and Node.js, a logger needs to be explicitly passed to the SDK that implements a very narrow Logger
interface defined by the Jaeger SDKs.
- Increasing the Logging in Backend Components
The jaeger-agent
and jaeger-collector
components provide useful debugging information when the log level is set to debug
. Every UDP packet that is received by jaeger-agent
is logged, as well as every batch that is sent by jaeger-agent
to jaeger-collector
. jaeger-collector
also logs every batch it receives and logs every span that is stored in the permanent storage.
- Configuring Tracing for Promtail
Tracing for Promtail can be configured using the tracing
block in the configuration file. This is currently limited to configuration per environment variables only.
- Monitoring Jaeger
Jaeger components only log to standard out, using structured logging library go.uber.org/zap
configured to write log lines as JSON encoded strings. The log level can be adjusted via the --log-level
command line switch; the default level is info
.
In conclusion, Jaeger-lib provides several options for internal logging that can be used to gather useful information for debugging. These options include setting the log level, using the logging reporter, increasing the logging in backend components, configuring tracing for Promtail, and monitoring Jaeger.
Sources:
- Jaeger documentation - Operator for Kubernetes
- Jaeger documentation - Troubleshooting
- Tremor - CNCF OpenTelemetry Jaeger Interworking
- Grafana Loki documentation - Configure Promtail
- Jaeger documentation - Monitoring
- Kubernetes blog - Boosting Kubernetes container runtime observability with OpenTelemetry
- GitLab documentation - Distributed tracing development guidelines
- Jaeger-lib code snippets