Summary of Open Issue: Increase sync-block-duration Default in Thanos
Issue Title: Increase sync-block-duration default
Created By: jpds on 2023-04-17
Overview:
The current default value for the Thanos store API’s --sync-block-duration is set to 3 minutes. Contributor jpds proposes increasing this value to 1 hour, citing that new blocks are typically shipped every 2 hours by sidecars and that the queriers manage the most recent metrics. jpds supports this request by sharing insights from their self-hosted S3 storage setup, illustrating the impact of the current duration on system performance.
Community Feedback:
- fpetkovski expresses agreement that the 3-minute setting is too aggressive, potentially leading to excessive load and costs on object storage. However, they caution that a 1-hour default might be too high, suggesting 15 minutes as a reasonable compromise to accommodate users with a maximum head block duration of 1 hour.
- matej-g concurs with the suggestion of 15 minutes, marking it as a sensible option.
- jpds later clarifies that, after experimenting with various sync durations, they find 1 hour and 3 minutes (1h3m) to be optimal. The additional 3 minutes accounts for synchronization scheduling, preventing conflicts when new blocks arrive at the top of the hour. They assert that 15 minutes could be too frequent for scenarios with multiple store APIs.
Additional Comment:
- yeya24 points out that
--sync-block-durationis configurable, emphasizing user flexibility but reiterates that 15 minutes appears to be an adequate default for most use cases.
Labels:
- component: store
- feature request/improvement
- good first issue
- help wanted
Conclusion:
The issue has garnered varying perspectives on the appropriate default setting for --sync-block-duration, suggesting a range of possibilities from 15 minutes to 1 hour and 3 minutes, with considerations for user configurations and potential performance impacts in object storage. Further discussion may be necessary to reach a consensus.
