[SERVER-59925] Stress test and fix deadlock in expiring idle buckets Created: 13/Sep/21  Updated: 29/Oct/23  Resolved: 01/Nov/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.2.0, 5.0.4, 5.1.0-rc3

Type: Bug Priority: Major - P3
Reporter: Gregory Noma Assignee: Gregory Noma
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-59888 Deadlock between BucketCatalog::_stat... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.1, v5.0
Sprint: Execution Team 2021-11-01, Execution Team 2021-11-15
Participants:

 Description   

When the time-series bucket catalog reaches its memory usage threshold, expires idle buckets in order to attempt to stay under this threshold. However, the only testing we currently have for this behavior is timeseries_idle_buckets.js. We can better exercise this logic, as well as the time-series insert logic in general, with a stress test which performs a large volume of inserts across multiple threads and with a large set of metaField values.



 Comments   
Comment by Githook User [ 02/Nov/21 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-59925 Stress test time-series inserts

(cherry picked from commit cf80c11bc5308d9b889ed61c1a3eeb821839df56)
Branch: v5.0
https://github.com/mongodb/mongo/commit/f3ca90647f8dac1aa9f0015a4ba5fe0e4b12b60e

Comment by Githook User [ 02/Nov/21 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-59925 Stress test time-series inserts

(cherry picked from commit cf80c11bc5308d9b889ed61c1a3eeb821839df56)
Branch: v5.1
https://github.com/mongodb/mongo/commit/bb2475deaf971065c2d29a1930fac9a8164a9cf0

Comment by Githook User [ 01/Nov/21 ]

Author:

{'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}

Message: SERVER-59925 Stress test time-series inserts
Branch: master
https://github.com/mongodb/mongo/commit/cf80c11bc5308d9b889ed61c1a3eeb821839df56

Comment by Gregory Noma [ 14/Sep/21 ]

louis.williams I think that would cause issues with our current tests expecting certain buckets to remain open throughout the test. However we could certainly lower the thresholds for any targeted stress testing, so that we hit those thresholds without having to use an enormous amount of data.

Comment by Louis Williams [ 14/Sep/21 ]

We should consider also lowering some of the bucket thresholds in all of our testing.

Generated at Thu Feb 08 05:48:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.