Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17194

Low Throughput for YCSB 50-50 workload with high client threads

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.0-rc7, 3.0.0-rc8, 3.0.0-rc9, 3.0.0-rc10
    • Component/s: Performance, Storage, WiredTiger
    • Storage Execution
    • Fully Compatible
    • ALL
    • Hide

      Updated
      YCSB: https://github.com/achille/YCSB

      Populate database with 2 M documents
      ycsb load mongodb -s -P workloads/workloada -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p mongodb.writeConcern=acknowledged -p mongodb.database=ycsb -p recordcount=2000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=256 -p insertretrycount=10 -p readretrycount=1 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p mongodb.url=mongodb://localhost:27017 -p fieldnameprefix=f -p maxexecutiontime=96000 -p mongodb.readPreference=primary -p fieldlength=10 -p reconnectiontime=1000 -p operationcount=200000000

      Run Workload A
      ./bin/ycsb run mongodb -s -P workloads/workloada -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p mongodb.writeConcern=acknowledged -p mongodb.database=ycsb -p recordcount=2000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=256 -p insertretrycount=10 -p readretrycount=1 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p mongodb.url=mongodb://localhost:27017 -p fieldnameprefix=f -p maxexecutiontime=96000 -p mongodb.readPreference=primary -p fieldlength=10 -p reconnectiontime=1000 -p operationcount=50000000

      Show
      Updated YCSB: https://github.com/achille/YCSB Populate database with 2 M documents ycsb load mongodb -s -P workloads/workloada -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p mongodb.writeConcern=acknowledged -p mongodb.database=ycsb -p recordcount=2000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=256 -p insertretrycount=10 -p readretrycount=1 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p mongodb.url=mongodb://localhost:27017 -p fieldnameprefix=f -p maxexecutiontime=96000 -p mongodb.readPreference=primary -p fieldlength=10 -p reconnectiontime=1000 -p operationcount=200000000 Run Workload A ./bin/ycsb run mongodb -s -P workloads/workloada -p workload=com.yahoo.ycsb.workloads.CoreWorkload -p mongodb.writeConcern=acknowledged -p mongodb.database=ycsb -p recordcount=2000000 -p exportmeasurementsinterval=30000 -p fieldcount=10 -p timeseries.granularity=100 -p threadcount=256 -p insertretrycount=10 -p readretrycount=1 -p ignoreinserterrors=true -p reconnectionthroughput=10 -p mongodb.url=mongodb://localhost:27017 -p fieldnameprefix=f -p maxexecutiontime=96000 -p mongodb.readPreference=primary -p fieldlength=10 -p reconnectiontime=1000 -p operationcount=50000000

      Running ycsb with high thread count leads to low WT throughput on the 50-50 workload (Workload A). Stand-alone mogod with wiredTiger storage engine.

      Seeing with that high levels of WT Rollback.

      Update:

      • Related to SERVER-16662 in that the problem requires many more threads than cores. Running with few cores has high performance. Running with many more thread than cores (256 on 12) leads to dramatic (50% or more) reduction in throughput.
      • Differs from SERVER-16662 in that the performance drop is steady (not drops to zero), and is correllated with wt transactions: transactions rolled back stat being the same level or higher than WT transaction: transactions completed.

      With 3.0.0 large number of documents in not required. Issue manifests with 2M documents.

      From the original description:

      In the graph below, before time A there are 256 client threads. After time B there are 32 threads. Throughput is higher with 32 threads. Substantial number of rollbacks with 256 threads.

      Results shown above from run with git version 7d9ec251cf0e70bc0f9bb246aacfb6e62226ad37

        1. 3.0build256.png
          155 kB
          David Daly
        2. patch.png
          153 kB
          David Daly
        3. Patch256.png
          166 kB
          David Daly
        4. workload50-50.png
          147 kB
          David Daly

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            david.daly@mongodb.com David Daly (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: