Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-87267

Increase bucket_catalog memory usage threshold for timeseries_dynamic_bucket_sizing.js

    • Storage Execution
    • Fully Compatible
    • v8.0, v7.3, v7.0
    • Execution Team 2024-04-29
    • 0

      timeseries_dynamic_bucket_sizing.js tests whether buckets are closed due to size or due to cache pressure. The first half of the test fills 1000 buckets to the point where they should roll over due to size, and checks that this is the case. The second half of the test fills 1500 buckets until they roll over due to cache pressure - 1500 buckets being, in this case, the cardinality of buckets necessary for the size limit for a bucket under cache pressure to be less than the general bucket size limit (buckets are closed when they hit the lesser of these two values).

      After the initial fix in SERVER-83377 , the test would fail rarely with the failure being that the count of buckets closed due to cache pressure was slightly fewer than expected. In each of these failures, the number of buckets that had been closed due to memory threshold was greater than 0. From running and experimenting with the test it looks like the catalog memory usage during the test reaches around 720MB+ locally, compared to the memory usage threshold of around ~800MB locally. The threshold is the threshold at which we start archiving, and then closing, buckets due to memory threshold. The threshold is calculated by default as ~2.5% of system memory (here). This value can differ on different machines and builds. Once a number of buckets is being closed due to memory threshold, it seems reasonable that this could impact the number of buckets closed due to other factors.

      After SERVER-88707 added an explicit check to this test that checks that the number of buckets closed due to memory threshold is 0, this test started failing more often. 

      This ticket increases the the memory usage threshold to be a fixed value that should be large enough to prevent catalog memory usage from hitting it. This allows the test to test the interplay of buckets being closed due to cache pressure versus size without buckets being closed due to memory threshold being thrown into the mix.

       

            Assignee:
            damian.wasilewicz@mongodb.com Damian Wasilewicz
            Reporter:
            damian.wasilewicz@mongodb.com Damian Wasilewicz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: