Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-4905

Throughput degradation captured by million collection test

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.3.1
    • Affects Version/s: None
    • Component/s: None
    • None
    • 5
    • Storage Engines 2019-08-12

      The million collection test running in Evergreen captured the throughput degradation while populating documents into a large number of collections. The avg throughput of the specific populating round was 6319 ops/s, lower than the target throughput 7500 ops/s.

      [2019/07/02 10:36:29.917] ======
      [2019/07/02 10:36:29.917] Starting a new run
      [2019/07/02 10:36:29.917] Start populating documents into collections ...
      [2019/07/02 10:36:29.917] Finish populating 687 documents into each of the collections
      [2019/07/02 10:36:29.917] Command for the run:
      [2019/07/02 10:36:29.917]   java -jar ../POCDriver/bin/POCDriver.jar -i 0 -u 100 -k 0 -q 10000 -z 687 -d 300 -y 58196 --collectionKeyMax 687 -o ../results/out.csv -t 64 -b 1
      [2019/07/02 10:50:41.476] Run completed avg latency with 58196 collections is 18.1029855315ms/op and throughput of 7659ops/sec
      [2019/07/02 10:50:41.476] The run completed with statistics below:
      [2019/07/02 10:50:41.476]   - avg response time (latency): 18.1029855315 (ms/op), avg throughput: 7659 (ops/sec)
      [2019/07/02 10:50:41.476]   - total run time: 8203.26333308, configured total_runtime: 86400
      [2019/07/02 10:50:41.476]   - num of collections: 58196, configured num_collections: 75000
      [2019/07/02 10:50:41.477]   - threads: 64, configured num_threads: 64
      [2019/07/02 10:50:41.477] ======
      [2019/07/02 10:50:41.477] Starting a new run
      [2019/07/02 10:50:41.477] Start populating documents into collections ...
      [2019/07/02 10:50:41.477] Finish populating 549 documents into each of the collections
      [2019/07/02 10:50:41.477] Command for the run:
      [2019/07/02 10:50:41.477]   java -jar ../POCDriver/bin/POCDriver.jar -i 0 -u 100 -k 0 -q 10000 -z 549 -d 300 -y 72745 --collectionKeyMax 549 -o ../results/out.csv -t 64 -b 1
      [2019/07/02 10:55:43.453] Run completed avg latency with 72745 collections is 30.3910063679ms/op and throughput of 6319ops/sec
      [2019/07/02 10:55:43.453] The run completed with statistics below:
      [2019/07/02 10:55:43.453]   - avg response time (latency): 30.3910063679 (ms/op), avg throughput: 6319 (ops/sec)
      [2019/07/02 10:55:43.453]   - total run time: 9055.63679218, configured total_runtime: 86400
      [2019/07/02 10:55:43.453]   - num of collections: 72745, configured num_collections: 75000
      [2019/07/02 10:55:43.453]   - threads: 64, configured num_threads: 64
      [2019/07/02 10:55:43.453] Average throughput is less than 7500.0 - test failed!
      [2019/07/02 10:55:43.453] Test Failed
      [2019/07/02 10:55:43.903] + exit_code=123 

            Assignee:
            luke.pearson@mongodb.com Luke Pearson
            Reporter:
            luke.chen@mongodb.com Luke Chen
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: