Production Monitoring with jaegertracing/jaeger-lib

Makefile Targets

When deploying jaeger-lib in production, it’s crucial to ensure that the library functions correctly and adheres to coding standards. This is typically managed using the Makefile provided in the repository. Key targets that facilitate this process include:

• install-dep: Installs the necessary dependencies.

  make install-dep
  

• test: Executes the test suite to confirm code integrity.

  make test
  

• lint: Static code analysis to catch potential issues.

  make lint
  

• test-and-lint: Combines both the test and lint operations. This is the default goal if no target is specified.

  make test-and-lint
  

Using these commands ensures that any code changes are validated before deployment, minimizing the risk of introducing bugs into a production environment.

Metrics Collection

Monitoring performance in production is achieved through metrics collection. The jaeger-lib provides a flexible metrics interface that can integrate with various backends, including Prometheus. The Backend struct within metrics/metricstest/local.go demonstrates how metrics are scoped and aggregated:

type Backend struct {
    cm         sync.Mutex
    gm         sync.Mutex
    tm         sync.Mutex
    hm         sync.Mutex
    counters   map[string]*int64
    gauges     map[string]*int64
    timers     map[string]*localBackendTimer
    histograms map[string]*localBackendHistogram
    stop       chan struct{}
    wg         sync.WaitGroup
    TagsSep    string
    TagKVSep   string
}

This structure allows efficient data aggregation in-vm, which can later be exported to monitoring systems.

Timer Implementation

To measure operation durations, the library uses a Timer interface, which allows for recording the duration of operations. For example:

type Timer interface {
    Record(time.Duration)
}

This can be implemented to track the time taken for specific requests or operations.

Prometheus Integration

To export metrics to a Prometheus backend, the library provides a Factory pattern to facilitate the creation of metric objects. Here’s a basic implementation of how to observe events with a timer:

func (t *timer) Record(v time.Duration) {
    t.histogram.Observe(float64(v.Nanoseconds()) / float64(time.Second/time.Nanosecond))
}

Using the Record method allows the application to record latency metrics, which can be scraped by Prometheus for monitoring.

Running Tests and Linting in CI

When integrating jaeger-lib into a continuous integration pipeline, it’s vital to set up testing and linting correctly. The Makefile supports these actions through the test-ci target, which can be invoked in a CI/CD setup:

make test-ci

By regularly running these commands, teams can ensure that any code modifications adhere to quality standards and do not degrade the performance or reliability of the system in production.

Release Process

To maintain seamless operations, it’s important to follow a structured release process. After verifying changes with testing and monitoring, a PR should be prepared with a conventional title:

Preparing release 2.1.0

After merging, ensure that recent changes are documented in CHANGELOG.md using:

git log --pretty=format:'- %s -- %an'

Following this, a release can be created on GitHub, effective communication of these changes is critical for distributed teams.

Conclusion

For effective production monitoring with jaegertracing/jaeger-lib, focus on utilizing the built-in tools for metrics and monitoring while adhering to the practices outlined above to ensure code quality and system reliability. Use the Makefile for building and testing, leverage the metrics interfaces for monitoring, and maintain a structured release process for continuous improvements.