[SERVER-69899] Dynamically size time series buckets based on workload and WT cache size Created: 22/Sep/22  Updated: 03/Nov/23  Resolved: 19/Oct/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.2.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Dan Larkin-York Assignee: Fausto Leyva (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-82798 Respect time-series bucket minimum si... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2022-10-17, Execution Team 2022-10-31
Participants:

 Description   

After SERVER-66690 and SERVER-66685, for high cardinality cases, we can run into WT cache pressure if we let buckets grow to the default size. In the case where we have a large number of active (open or archived) buckets and we allow the full default bucket size (125KiB), we could easily exceed the WT cache size and get into a state where each document write will trigger a write to disk.

Instead, we should track the number of active buckets and lower the size limit dynamically so that all of the active buckets take up no more than half the WT cache. This will generally prevent us from getting into a constant write-through pattern, at the cost of slightly worse disk usage and query performance (due to smaller buckets). This offers the users at trade-off between instance size (RAM) and disk size.



 Comments   
Comment by Githook User [ 18/Oct/22 ]

Author:

{'name': 'Faustoleyva54', 'email': 'fausto.leyva@mongodb.com', 'username': 'Faustoleyva54'}

Message: SERVER-69899 Dynamically size time series buckets based on workload and WT cache size
Branch: master
https://github.com/mongodb/mongo/commit/6eec84874c81a559073f17a17f47c7b2551d64c6

Generated at Thu Feb 08 06:14:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.