Shoulder.dev Logo Shoulder.dev

Deduplication and Merging - thanos-io/thanos

Thanos is an open-source project that provides scalable Prometheus monitoring through a distributed and highly available architecture. It supports deduplication and merging of metrics, which are essential features for managing large-scale monitoring systems. This explanation will cover the possible options and provide examples for each option, as described in the provided documentation and code snippets.

Deduplication

Deduplication in Thanos is the process of removing redundant data points based on their label sets. This is particularly useful when dealing with high availability (HA) setups, where multiple Prometheus instances might scrape the same targets.

Query-time Deduplication

Thanos supports query-time deduplication using the --query.replica-label flag in the thanos query command. This flag specifies the label to treat as a replica indicator for deduplication purposes. By default, Thanos uses a naive algorithm for merging blocks, which works well for deduplication of blocks with precisely the same samples, like those produced by Receiver replication. However, if you need a different deduplication algorithm, you can set it via the --deduplication.func flag.

Example:

thanos query \
  --http-address        0.0.0.0:19192 \
  --endpoint            1.2.3.4:19090 \
  --endpoint            1.2.3.5:19090 \
  --query.replica-label replica

Vertical Compaction

Thanos and Prometheus support vertical compaction, which is the process of compacting multiple streams of blocks into one. This feature is useful for backfilling and offline deduplication of series.

In Prometheus, vertical compaction can be triggered by setting a hidden flag and putting additional TSDB blocks in Prometheus' local data directory. In Thanos, this works similarly, but on a larger scale and using external labels for grouping.

Thanos supports two deduplication algorithms for vertical compaction:

  1. Naive: This algorithm works well for deduplication of blocks with precisely the same samples, like those produced by Receiver replication.
  2. Penalty: This algorithm is the same as the online deduplication of the querier, except for counter reset handling. It is useful for deduplicating blocks with different samples, like those produced by Prometheus replicas.

To enable the penalty-based deduplication algorithm, you can use the --deduplication.func=penalty flag in the thanos compact command.

Example:

thanos compact \
  --data-dir /path/to/data \
  --deduplication.func penalty

Merging

Merging in Thanos is the process of combining metrics from different sources based on their label sets. This is useful for creating a global view of your monitoring system.

Global View

Thanos supports a global view of your monitoring system using the thanos query command with the --meta flag. This flag specifies the location of the Thanos sidecar or Prometheus instance that contains the global external labels.

Example:

thanos query \
  --http-address        0.0.0.0:19192 \
  --meta                1.2.3.4:9090

Deduplication Replica Labels

Thanos supports deduplication of replica labels using the --query.replica-label flag in the thanos query command. This flag specifies the label to treat as a replica indicator for deduplication purposes.

Example:

thanos query \
  --http-address        0.0.0.0:19192 \
  --endpoint            1.2.3.4:19090 \
  --endpoint            1.2.3.5:19090 \
  --query.replica-label replica

Conclusion

Thanos provides powerful deduplication and merging capabilities for managing large-scale monitoring systems. By using the provided flags and commands, you can configure Thanos to meet your specific needs and optimize your monitoring infrastructure.