Thanos is an open-source CNCF Sandbox project that extends Prometheus to create a global-scale, highly available monitoring system. It integrates with existing Prometheus deployments and leverages Prometheus’s native storage format. This section will cover the possible options for integrating Thanos with Prometheus and provide examples for each option.
Option 1: Thanos Sidecar
The Thanos Sidecar is a method to integrate Thanos with Prometheus by deploying a Thanos Sidecar process alongside an existing Prometheus server. The Sidecar automatically scrapes metrics from Prometheus and stores them in a distributed object store, such as Amazon S3 or Google Cloud Storage.
Here’s an example of a Thanos Sidecar configuration:
# sidecar.yaml
graceful_shutdown_timeout: 1m
replication_factor: 1
store:
storegateway:
address: http://thanos-storegateway:10902
prometheus:
url: http://localhost:9090
Option 2: Thanos Compact Object Store
Thanos Compact Object Store (COS) is a method to integrate Thanos with Prometheus by using a separate Prometheus server that writes to a distributed object store. The Thanos COS periodically compacts and downsamples the data in the object store, reducing the storage requirements.
Here’s an example of a Thanos COS configuration:
# compact.yaml
replication_factor: 1
store:
object_store:
config:
# object store configuration
compact:
- job_name: 'prometheus'
relabel_configs:
- action: keep
regex: __name__ =~ "^(prometheus_)"
source_labels: [__name__]
min_time: 1h
max_time: 1d
time_range: 1d
Option 3: Thanos Query
Thanos Query is a method to integrate Thanos with Prometheus by deploying a separate Thanos Query process that can query multiple Prometheus servers and the distributed object store. The Thanos Query process can be used to query data from multiple Prometheus servers and the object store, providing a global view of the metrics.
Here’s an example of a Thanos Query configuration:
# query.yaml
graceful_shutdown_timeout: 1m
qurey:
- url: http://prometheus-server-1:9090
- url: http://prometheus-server-2:9090
- url: http://thanos-storegateway:10902
These are the possible options for integrating Thanos with Prometheus. The choice of which option to use depends on the specific use case and requirements.
Sources:
- Thanos Integration with Prometheus and Alertmanager: https://docs.bitnami.com/kubernetes/infrastructure/thanos/administration/enable-metrics
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.36/thanos/changelog.md
- Boom Table plugin for Grafana | Grafana Labs: https://grafana.com/grafana/plugins/yesoreyeram-boomtable-panel
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.36/contributing/community.md
- What recent optimizations in the Prometheus storage engine, TSDB, will enable in the future | Grafana Labs: https://grafana.com/blog/2020/07/27/what-recent-optimizations-in-the-prometheus-storage-engine-tsdb-will-enable-in-the-future
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.36/thanos/design.md
- Prometheus Monitoring OSS | Store large amounts of metrics: https://grafana.com/oss/prometheus
- A Look Inside GitLab’s Public Dashboards | Grafana Labs: https://grafana.com/blog/2019/06/24/a-look-inside-gitlabs-public-dashboards
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.21/proposals/config.md
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.36/thanos/getting-started.md
- Thanos - Highly available Prometheus setup with long term storage capabilities: https://thanos.io/v0.11/config.md