Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7293

Validate a window of history that spans 24 hours

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 5.1 Required
    • Affects Version/s: None
    • Component/s: None
    • None
    • 13

      One of the goals of durable history was to enable MongoDB to sustain a "normal" workload, while running a point-in-time scan for 24 hours.

      We should craft a test against MongoDB that does that and run it to ensure we have met the goal.

      The workload should:

      • Populate a database with 5 collections, in the order of 50GB of data spread across those collections.
      • Sustain a rate of new inserts and an equivalent rate of deletes. Update existing documents. The rate of these operations should remain within the capacity of the provisioned system. I recommend 100 inserts/deletes per second, and 100 updates.
      • Run the test on a 3 node replica set.
      • Update the MongoDB configuration settings to allow for a point-in-time scan that runs for a day (talk to alex.cameron to find out the options needed to do that).
      • Implement a scan query that runs for a day, it should be rate limited, and cycle through reading the dataset. Probably at ~10 operations/second.

      Talk to brian.lane about how to determine whether the workload is a success, but my recommendation is to check for stable throughput at the expected rate, along with no throughput stalls.

        1. ycsb_output_txn.log
          1.45 MB
        2. ycsb_output_notxn.log
          1.44 MB
        3. Update.zoom.2021-05-24.gif
          Update.zoom.2021-05-24.gif
          19 kB
        4. Update.histogram.2021-05-24.gif
          Update.histogram.2021-05-24.gif
          20 kB
        5. Update.5minavg.jpg
          Update.5minavg.jpg
          89 kB
        6. Update.5min.compare.jpg
          Update.5min.compare.jpg
          102 kB
        7. Update.2021-05-24.gif
          Update.2021-05-24.gif
          38 kB
        8. Screen Shot 2021-06-02 at 4.21.11 PM.png
          Screen Shot 2021-06-02 at 4.21.11 PM.png
          65 kB
        9. Screen Shot 2021-06-01 at 7.28.35 PM.png
          Screen Shot 2021-06-01 at 7.28.35 PM.png
          72 kB
        10. Screen Shot 2021-06-01 at 7.19.19 PM.png
          Screen Shot 2021-06-01 at 7.19.19 PM.png
          60 kB
        11. Screen Shot 2021-05-26 at 10.01.30 am.png
          Screen Shot 2021-05-26 at 10.01.30 am.png
          113 kB
        12. Read.5minavg.jpg
          Read.5minavg.jpg
          116 kB
        13. Read.5min.compare.jpg
          Read.5min.compare.jpg
          101 kB
        14. Read.10secavg.jpg
          Read.10secavg.jpg
          97 kB
        15. Insert.5minavg.jpg
          Insert.5minavg.jpg
          65 kB
        16. Insert.5min.compare.jpg
          Insert.5min.compare.jpg
          61 kB
        17. image.jpeg
          image.jpeg
          101 kB
        18. FTDC.Screenshot.2021-05-24.png
          FTDC.Screenshot.2021-05-24.png
          396 kB
        19. ftdc.5000.tgz
          4.21 MB
        20. FTDC.2500x24.tgz
          53.96 MB
        21. FTDC.2500x24.notxn.tgz
          47.22 MB
        22. ftdc.2500.tgz
          2.96 MB
        23. ftdc.2021-05-24.tgz
          18.27 MB
        24. ftdc.1250.tgz
          2.85 MB

            Assignee:
            keith.smith@mongodb.com Keith Smith
            Reporter:
            alexander.gorrod@mongodb.com Alexander Gorrod
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: