Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4614

Trigger sweep immediately to reclaim space after a table drop

    XMLWordPrintable

Details

    • Task
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • WT3.2.0, 4.1.10, 4.0.24
    • None
    • 3
    • Storage Engines 2019-03-25, Storage Engines 2019-04-08
    • v4.0

    Description

      After the WiredTiger import in MongoDB (0c6ba8d8be02dd34a46c3e9533971f1739b6ad8e), WT's cache while running as an in-memory storage engine does not reconcile memory promptly causing the test to fail with a `WT_CACHE_FULL` error.
      A build failure sprung up as a consequence of this, where it requires approximately 5-6GB of cache to pass successfully, as opposed to ~3GB prior to the WT import.

      I'm attaching diagnostic data I collected of the test (agg_sort_external.js) from before and after the WT import, as well as some graphs that could be useful to understanding the problem.
      From my observations, after the import, the WT cache fill ratio is climbing steadily after a certain point and the tracked bytes belonging to leaf pages in the cache climbs towards the 4GB cache limit, causing the test to fail.
      Prior to the import, the WT cache fill ratio would oscillate around 50%, as well as the tracked bytes belonging to leaf pages in the cache at around 2GB.

      `metrics.passing` was taken from `970e0c02de8c5922efcab74cfee258f447f94741` and `metrics.failing` was taken from `83336cb56b269195110253918d226cbba4377a03` on the MongoDB repo.

      Attachments

        1. failing-1.png
          368 kB
          Gregory Wlodarek
        2. failing-2.png
          342 kB
          Gregory Wlodarek
        3. failing-3.png
          55 kB
          Gregory Wlodarek
        4. metrics.failing
          47 kB
          Gregory Wlodarek
        5. metrics.passing
          39 kB
          Gregory Wlodarek
        6. passing-1.png
          371 kB
          Gregory Wlodarek
        7. passing-2.png
          352 kB
          Gregory Wlodarek
        8. passing-3.png
          65 kB
          Gregory Wlodarek
        9. runs_evg.png
          176 kB
          Sulabh Mahajan
        10. wt_runs.png
          196 kB
          Sulabh Mahajan

        Issue Links

          Activity

            People

              sulabh.mahajan@mongodb.com Sulabh Mahajan
              gregory.wlodarek@mongodb.com Gregory Wlodarek
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: