Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7619

Add a new optimization to skip pages in cursor traversal when all entries on the page are deleted

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 4.4.8, 5.0.2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • 8
    • Storage - Ra 2021-06-14, Storage - Ra 2021-06-28, Storage - Ra 2021-07-12, Storage - Ra 2021-07-26

      This is a break-off ticket fromĀ WT-6796 as we want to understand why delete throughout decreases as some of the remove-only benchmarks progress.

      There are few things to consider here:

      • As discussed inĀ WT-6796, the reduced delete operation doesn't seem to correlate with history store file size.
      • The delete throughout is decreasing as we keep deleting random documents from the test collections without replacing them.
      • The benchmark is specifically testing system performance with long running transactions. From the plot above we can notice that the stat "range of timestamps currently pinned" is growing to 1 hour.
      • Deleting records in WT essentially entails adding a stop timestamp the records. The actual record in not removed until the stop time becomes globally visible.
      • We need to gather more information about the benchmark. For example, how cursors are being used to delete the data and if there any indexes on the test collection.

      The aim of this ticket is to find the root cause behind the delete performance degradation and create tickets for possible improvements.

        1. FinalChangesOneHourPhase.png
          FinalChangesOneHourPhase.png
          348 kB
        2. full-range.png
          full-range.png
          48 kB
        3. full-range-benchmark.png
          full-range-benchmark.png
          47 kB
        4. hist-collection0.png
          hist-collection0.png
          61 kB
        5. hist-half-range.png
          hist-half-range.png
          24 kB
        6. image-2021-05-31-16-55-40-718.png
          image-2021-05-31-16-55-40-718.png
          593 kB
        7. image-2021-06-10-15-00-55-476.png
          image-2021-06-10-15-00-55-476.png
          425 kB
        8. image-2021-06-11-09-58-12-690.png
          image-2021-06-11-09-58-12-690.png
          342 kB
        9. image-2021-06-11-09-59-21-306.png
          image-2021-06-11-09-59-21-306.png
          483 kB
        10. image-2021-06-11-10-24-38-903.png
          image-2021-06-11-10-24-38-903.png
          566 kB
        11. image-2021-06-11-10-31-36-408.png
          image-2021-06-11-10-31-36-408.png
          608 kB
        12. image-2021-06-11-10-35-23-972.png
          image-2021-06-11-10-35-23-972.png
          681 kB
        13. image-2021-06-15-14-02-10-227.png
          image-2021-06-15-14-02-10-227.png
          373 kB
        14. image-2021-06-15-14-03-19-961.png
          image-2021-06-15-14-03-19-961.png
          377 kB
        15. image-2021-06-15-14-50-04-635.png
          image-2021-06-15-14-50-04-635.png
          410 kB
        16. image-2021-06-23-11-06-43-745.png
          image-2021-06-23-11-06-43-745.png
          453 kB
        17. image-2021-07-01-17-01-08-037.png
          image-2021-07-01-17-01-08-037.png
          163 kB
        18. image-2021-07-02-21-45-26-020.png
          image-2021-07-02-21-45-26-020.png
          346 kB
        19. LatestOptimization_10min_window.png
          LatestOptimization_10min_window.png
          366 kB
        20. optimization-partial.png
          optimization-partial.png
          271 kB
        21. RemoveGtc.png
          RemoveGtc.png
          67 kB
        22. RemoveLtc.png
          RemoveLtc.png
          60 kB
        23. skipped-by-cursor-next-new.png
          skipped-by-cursor-next-new.png
          383 kB
        24. slow-delete.png
          slow-delete.png
          391 kB
        25. workgen-new-benchmark.png
          workgen-new-benchmark.png
          275 kB

            Assignee:
            haseeb.bokhari@mongodb.com Haseeb Bokhari (Inactive)
            Reporter:
            haseeb.bokhari@mongodb.com Haseeb Bokhari (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: