Insert performance drop in MongoDB 6.0.14 compared to 4.4.29

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Server Triage
    • ALL
    • Hide
      • 4.4.29 version: https://github.com/mongodb/mongo/tree/r4.4.29
      • 6.0.14 version: https://github.com/mongodb/mongo/tree/r6.0.14
      • ycsb version
      • ycsb benchmark worload config: both with write_concern specified with w = 1
        recordcount=100000000
        #operationcount=100000000
        operationcount=100000000
        workload=com.yahoo.ycsb.workloads.CoreWorkload
        threadcount = 100readallfields=true
        readproportion=0
        updateproportion=0
        scanproportion=0
        insertproportion=1
        readmodifywriteproportion=0
        requestdistribution=zipfian
        insertorder = hashed
        readallfields=true
        fieldlength = 200
        fieldcount = 10
        mongodb.writeConcern = acknowledged
        mongodb.database = ycsb
        mongodb.url = mongodb://pwd@host/admin?&w=1
        table = ycsb  

        **

      Show
      4.4.29 version:  https://github.com/mongodb/mongo/tree/r4.4.29 6.0.14 version:  https://github.com/mongodb/mongo/tree/r6.0.14 ycsb version ycsb benchmark worload config:  both with write_concern specified with w = 1 recordcount=100000000 #operationcount=100000000 operationcount=100000000 workload=com.yahoo.ycsb.workloads.CoreWorkload threadcount = 100readallfields= true readproportion=0 updateproportion=0 scanproportion=0 insertproportion=1 readmodifywriteproportion=0 requestdistribution=zipfian insertorder = hashed readallfields= true fieldlength = 200 fieldcount = 10 mongodb.writeConcern = acknowledged mongodb.database = ycsb mongodb.url = mongodb: //pwd@host/admin?&w=1 table = ycsb **
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Before upgrading our MongoDB cluster version from 4.4.29 to 6.0.14,  we did some ycsb benchmarking to this two versions.  Benchmak result shows that under mixed read/write and pure insert workloads,  MongoDB 6.0.14 suffers a 15% performance drop compared to 4.4.29, currenty we cannot find out  the reason for such a high performance drop, so it will be very helpful if MongoDB Team can help us figure it out. Thanks.

       

      Following is one of our benchmarking details under ycsb pure insert workloads:

      • System Environments:

                     Operating System: Debian 10.13

                     CPU:  Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz(6 cores)

                     Memory: 50 GB

                     Disk:  1.92 TB SSD S4500

      • Benchmark Result: 4.4.29 27705 ops vs 6.0.14  23700 ops(attchment 1.png and 2.png)
      • WiredTiger Perf Stats:  db.serverStatus().wiredTiger.perf shows that 6.0.14 has 10% insert ops with write latency greater than 100 us, while 4.4.29 only has 5% of insert ops(total 1,0000,0000 ops)(attachment 6.0.14_perf.png and 4.4.29_perf.png){}
      • mongod Perf Stats:  linux-perf of mongod shows that mongod under 4.4.29 spends about 6.8% more in performInserts call, but code changes a lot from 4.4.29 to 6.0.14, and we cannot find out a main reason to explain the drop(attachment mongod_6.0.14_perf.png and mongod_4.4.29_perf.png)

        1. 1.png
          32 kB
          Song Zhus
        2. 2.png
          33 kB
          Song Zhus
        3. 4.4.29_perf.png
          51 kB
          Song Zhus
        4. 4.4.29_standalone_pure_insert.png
          129 kB
          Song Zhus
        5. 6.0.14_perf.png
          51 kB
          Song Zhus
        6. 6.0.14_standalone_pure_insert.png
          125 kB
          Song Zhus
        7. mongod_4.4.29_perf.png
          100 kB
          Song Zhus
        8. mongod_6.0.14_perf.png
          110 kB
          Song Zhus

            Assignee:
            Chris Kelly
            Reporter:
            Song Zhus
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: