[SERVER-20623] 10% Performance Regression in update path between 3.1.2 and master (Update.IncFewDocs, Update.IncNoIndex, Update.IncWithIndex, MultiUpdate.Uncontended.TwoDocs.NoIndex...) Created: 11/Sep/15  Updated: 06/Dec/22  Resolved: 17/Nov/16

Status: Closed
Project: Core Server
Component/s: Performance, Write Ops
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: David Daly Assignee: Backlog - Storage Execution Team
Resolution: Incomplete Votes: 0
Labels: mpreg
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
Related
Assigned Teams:
Storage Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:
Linked BF Score: 0

 Description   

Mongo-perf shows a large regression on all of the update tests sometime between 3.1.2 and 3.1.4 on wiredTiger, probably over multiple commits, and another ~4% drop since 3.1.4. On WiredTiger and MMAPv1, although the regression shows earlier on wiredTiger.

WiredTiger Results
MMAPv1 Results

Two simplified shell based reproductions.

Update.MultiUpdate.Uncontended.TwoDocs.NoIndex

var setupTestUncontendedTwoDocs = function( collection ) {
    collection.drop();
    var docs = [];
    for ( var i = 0; i < 4800; i++ ) {
        docs.push( { _id : i , x : 0 } );
    }
    collection.insert(docs);
    collection.getDB().getLastError();
};
use test0
setupTestUncontendedTwoDocs(db.MultiUpdate_Uncontended_TwoDocs_NoIndex0)
benchRun({"ops":[{"op":"update","multi":true,"query":{"_id":{"$in":[{"#RAND_INT_PLUS_THREAD":[0,100]},{"#RAND_INT_PLUS_THREAD":[0,100]}]}},"update":{"$inc":{"x":1}},"ns":"test0.MultiUpdate_Uncontended_TwoDocs_NoIndex0","safe":false,"w":0,"j":false,"writeCmd":true}],"seconds":5,"host":"127.0.0.1:27017","parallel":8})

Update.FieldAtOffset

var setup = function( collection ) {
                  collection.drop();
 
                  var kFieldCount = 512;
 
                  // Build the document and insert several copies.
                  var toInsert = {};
                  for (var i = 0; i < kFieldCount; i++) {
                      toInsert["a_" + i.toString()] = "a";
                  }
                  
                  var docs = [];
                  for (var i = 0; i < 100; i++) {
                      docs.push(toInsert);
                  }
                  collection.insert(docs);
                  collection.getDB().getLastError();
              }
use test0
setup(db.Update_FieldAtOffset0)
benchRun({"ops":[{"op":"update","multi":true,"query":{},"update":{"$set":{"a_256":"a"}},"ns":"test0.Update_FieldAtOffset0","safe":false,"w":0,"j":false,"writeCmd":true},{"op":"update","multi":true,"query":{},"update":{"$set":{"a_256":"aa"}},"ns":"test0.Update_FieldAtOffset0","safe":false,"w":0,"j":false,"writeCmd":true}],"seconds":5,"host":"127.0.0.1:27017","parallel":8})



 Comments   
Comment by Eric Milkie [ 17/Nov/16 ]

While it would be desirable to understand what exactly caused this particular slowdown, I don't think it's a reasonable use of resources to spend on it.
Going forward, we will be more proactive about investigating perf regressions as they are detected.

Comment by Githook User [ 11/Dec/15 ]

Author:

{u'username': u'dalyd', u'name': u'dalyd', u'email': u'david.daly@mongodb.com'}

Message: Overrides for SERVER-20623 for linux-wt-repl
Branch: mc-test
https://github.com/10gen/dsi/commit/9f0382cd368bb6153bb089ab3016d3366ced41eb

Comment by David Daly [ 30/Nov/15 ]

With recent commits between rc3 and rc4, this regression is less than 5% in general for standalone
Current MMAP results
and current WT results
Ignore the doc validation data in there, as we use a different comparison point for them.

However the WT with replication still shows a drop from 3.1.2, that occurred between 3.1.2 and 3.1.4. Results here

Comment by David Daly [ 24/Nov/15 ]

Also MultiUpdate.Uncontended.TwoDocs.NoIndex shows on 1Node MMAP in similar pattern.

Comment by Githook User [ 26/Oct/15 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'ksuarz@gmail.com'}

Message: SERVER-20623 overrides for update tests

This overrides the performance regression in the update tests for WT standalone
and MMAP standalone.

Signed-off-by: David Daly <david.daly@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/b80acee146f87d81648212fa92fb9c440bcf429f

Comment by Chung-yen Chang [ 14/Oct/15 ]

Just checked the latest data against 3.1.2 and we are still regressed.

mmapv1
wiredTiger

The worst regressions are for doc validation updates. For both storage engines, we saw close to 20% of regressions for the updates with doc validation. The other cases are seeing roughly 10% for wiredTiger and ~5% for mmapv1.

Comment by Chung-yen Chang [ 17/Sep/15 ]

This is a regression that happened quite a while back. Moving it into the CAP project for proper investigation.

Generated at Thu Feb 08 03:54:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.