[SERVER-20193] Performance regression in Single-threaded MMS workloads from git: a7338c4 Created: 28/Aug/15  Updated: 07/Dec/15  Resolved: 11/Sep/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Bug Priority: Major - P3
Reporter: Chung-yen Chang Assignee: David Hows
Resolution: Done Votes: 0
Labels: mpreg
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by WT-1973 MongoDB changes for WiredTiger 2.7.0 Closed
Duplicate
is duplicated by SERVER-20195 Performance regression in WiredTiger ... Closed
Related
is related to SERVER-19445 Performance regression in Mongo-Perf ... Closed
is related to WT-2066 Update the oldest transaction ID from... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

Big performance drop (>80%) from git: a7338c4 on single-threaded MMS workloads. Same build was marked red for Geo and Misc cases but there only one test failed for single-threaded cases and the drop was much smaller. I think it's safe to ignore that and just focus on the MMS cases.

The commit that caused this was a wiredTiger check-in. WT team is aware of this drop so I am putting WT as the component for this ticket so they can pick this ticket up.



 Comments   
Comment by Githook User [ 16/Sep/15 ]

Author:

{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

Message: Merge pull request #2188 from wiredtiger/visible_all_call

SERVER-20193 Fix obsolete transaction check
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/8be547b318f0195d0c3c13a692589631f381f935

Comment by Githook User [ 16/Sep/15 ]

Author:

{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

Message: SERVER-20193 Fix clearing of obsolete check transaction.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/0b359e0a3661f859db8d5d86c255045582f5d556

Comment by Githook User [ 10/Sep/15 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Merge pull request #2177 from wiredtiger/SERVER-20193

SERVER-20193 - Performance regression fix
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/b044603cf9c3032526ddc144fdf23ef9d862b617

Comment by Githook User [ 10/Sep/15 ]

Author:

{u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

Message: Merge pull request #2177 from wiredtiger/SERVER-20193

SERVER-20193 - Performance regression fix
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/b044603cf9c3032526ddc144fdf23ef9d862b617

Comment by Githook User [ 10/Sep/15 ]

Author:

{u'username': u'daveh86', u'name': u'David Hows', u'email': u'howsdav@gmail.com'}

Message: SERVER-20193 - Change when WT_TXN_NONE is set, update comments
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/a60a16cbf14f67c4b6a4c4452782f5faac1617cc

Comment by Githook User [ 10/Sep/15 ]

Author:

{u'username': u'daveh86', u'name': u'David Hows', u'email': u'howsdav@gmail.com'}

Message: SERVER-20193 - Performance regression fix
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/713daf2a75a6c602e9ffa23f74dfec2f0ddc9a8a

Comment by Michael Cahill (Inactive) [ 07/Sep/15 ]

david.hows, I'm pretty sure my comment above explains what's going on here – can you please re-run this and make sure you can see the slowdown?

The fix is to call __wt_txn_update_oldest more frequently in this workload, for some definition of "more frequently", so that obsolete updates can be freed before the page size hits memory_page_max.

For example, we change the end of __wt_update_serial in btree.i to something like this:

if (upd->next == NULL)
    return (0);
 
if (page->modify->obsolete_check_txn != WT_TXN_NONE) {
    if (!__wt_txn_visible_all(session, page->modify->obsolete_check_txn)) {
        /* Check if the oldest ID can move forward and re-check. */
        __wt_txn_update_oldest(session);
        if (!__wt_txn_visible_all(session, page->modify->obsolete_check_txn))
            return (0);
}
 
F_CAS_ATOMIC(page, WT_PAGE_RECONCILIATION, ret);
/* If we can't lock it, don't scan, that's okay. */
if (ret != 0)
    return (0);
obsolete = __wt_update_obsolete_check(session, page, upd->next);
...

Does that make sense?

Comment by Michael Cahill (Inactive) [ 29/Aug/15 ]

https://evergreen.mongodb.com/task/performance_linux_wt_repl_singleThreaded_a7338c436160e259d0b20058f0a9ae4b442ca102_15_08_28_06_06_32

This is the same issue as SERVER-19445 – if the oldest transaction ID tracked by WiredTiger falls behind, workloads that do many updates to medium-large documents to hit forced eviction. The regression was introduced by WT-2066.

Generated at Thu Feb 08 03:53:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.