Thanos is an open-source project that extends Prometheus to provide a highly available, scalable, and long-term metric storage system. It enables unlimited retention of metrics by leveraging object storage solutions such as GCP, S3, Azure, Swift, and Tencent COS. By default, there is no retention set for object storage data, meaning that you store data forever. However, you can configure retention using --retention.resolution-raw, --retention.resolution-5m, and --retention.resolution-1h flags. Not setting them or setting to 0s means no retention.
Thanos stores data in blocks, which are immutable directories containing sample data and persisted indices. The compactor, a singleton process, consolidates multiple smaller blocks into larger ones, reducing the total storage size and load on store nodes. It also downsamples and applies retention policies during this process.
There are several options to configure retention in Thanos, depending on your use case:
- No retention: By not setting any retention flags or setting them to
0d, samples at all resolution levels will be kept forever.
Example:
--retention.resolution-raw=0d --retention.resolution-5m=0d --retention.resolution-1h=0d
- Specific retention per resolution: You can set specific retention periods for each resolution.
Example:
--retention.resolution-raw=1d --retention.resolution-5m=7d --retention.resolution-1h=30d
This configuration will keep raw resolution data for 1 day, 5-minute resolution data for 7 days, and 1-hour resolution data for 30 days.
Please note that blocks are only deleted after they completely “fall off” of the specified retention policy. In other words, the “max time” of a block needs to be older than the amount of time you had specified.
Sources:
