Thanos, an open-source CNCF Sandbox project, extends Prometheus to create a global-scale, highly available monitoring system. It supports various object stores for primary storage of metrics and metadata. This document explains how to configure object storage and the data layout and format for primary Thanos components that are “block” aware.
Object Storage Support
Thanos supports any object stores that can be implemented against the thanos/objstore package’s Bucket interface. The following object storage client implementations are currently available:
| Provider | Maturity | Aimed For | Auto-tested on CI | Maintainers |
|---|---|---|---|---|
| Google Cloud Storage | Stable | Production Usage | yes | @bwplotka |
| AWS/S3 (and all S3-compatible storages e.g disk-based Minio) | Stable | Production Usage | yes | @bwplotka |
| Azure Storage Account | Stable | Production Usage | no | @vglafirov |
| OpenStack Swift | Beta (working PoC) | Production Usage | yes | @FUSAKLA |
| Tencent COS | Beta | Production Usage | no | @jojohappy,@hanjm |
| AliYun OSS | Beta | Production Usage | no | @shaulboozhiao,@wujinhu |
| Local Filesystem | Stable | Testing and Demo only | yes | @bwplotka |
Configuring Object Storage
To configure object storage, you can use a configuration file or put the YAML config directly using the --objstore.config-file or --objstore.config flags.
Example configuration for Google Cloud Storage:
type: GCS
config:
bucket: "thanos-bucket"
prefix: "blocks/"
insecure_skip_verify: false
disable_compression: false
For more information on configuring specific object storage providers, refer to the official documentation.
Data Layout and Format
The data layout and format for primary Thanos components that are “block” aware (sidecar, compact, receive, and store gateway) are as follows:
- Metrics are stored in separate blocks in the object storage.
- Each block is a directory containing the following:
meta.json: Metadata about the block, including the block range, labels, and chunk data.chunks/: A directory containing chunk files, which are compressed and encoded metric data.
Instance Principal Provider
For example, configuring the OCI Instance Principal Provider:
type: OCI
config:
provider: "instance-principal"
bucket: ""
compartment_ocid: "<compartment_ocid>"
Raw Provider
For example, configuring the OCI Raw Provider:
type: OCI
config:
provider: "raw"
bucket: "<bucket_name>"
compartment_ocid: "<compartment_ocid>"
tenancy_ocid: "<tenancy_ocid>"
user_ocid: "<user_ocid>"
region: "<region>"
fingerprint: "<fingerprint>"
privatekey: "<private_key>"
passphrase: "<passphrase>"
Using Object Storage with Thanos Components
Sidecar
The sidecar component fetches blocks from object storage and serves them via a local HTTP server.
Example configuration:
type: GCS
config:
bucket: "thanos-bucket"
prefix: "blocks/"
object_config:
insecure_skip_verify: false
disable_compression: false
Compact
The compact component compacts blocks and stores them in object storage.
Example configuration:
type: GCS
config:
bucket: "thanos-bucket"
prefix: "blocks/"
object_config:
insecure_skip_verify: false
disable_compression: false
Receive
The receive component receives blocks from external sources and stores them in object storage.
Example configuration:
type: GCS
config:
bucket: "thanos-bucket"
prefix: "blocks/"
object_config:
insecure_skip_verify: false
disable_compression: false
Store Gateway
The store gateway component serves blocks from object storage via a local HTTP server.
Example configuration:
type: GCS
config:
bucket: "thanos-bucket"
prefix: "blocks/"
object_config:
insecure_skip_verify: false
disable_compression: false
Conclusion
Thanos supports various object storage providers, allowing you to set up a highly available Prometheus monitoring system with long-term storage capabilities. By understanding the data layout and format and configuring object storage correctly, you can efficiently use Thanos components with object storage.
