Details
-
Type:
Improvement
-
Status: Closed
-
Priority:
Major - P3
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: WT2.6.0
-
Labels:None
Description
At the end of update_serial we check for obsolete update structures. If we can't evict b/c of a checkpoint and that list grows large, this is wasted work inside a transaction.
Updates shouldn't be blocked, that is, only a single thread at a time is doing the check for obsolete update structures.
It would be pretty easy to turn off the check for obsolete update structures when checkpoint is running on this file; let's explore ways to tighten it down further using the checkpoint generation and other transactional information.
Activity
Suggested change:
diff --git a/src/include/serial.i b/src/include/serial.i
|
index e3581ae..afb01b3 100644
|
--- a/src/include/serial.i
|
+++ b/src/include/serial.i
|
@@ -255,7 +255,7 @@ __wt_update_serial(WT_SESSION_IMPL *session, WT_PAGE *page,
|
* obsolete check at a time, and to protect updates from disappearing
|
* under reconciliation.
|
*/
|
- if (upd->next != NULL) {
|
+ if (!S2BT(session)->checkpointing && upd->next != NULL) {
|
F_CAS_ATOMIC(page, WT_PAGE_SCANNING, ret);
|
/* If we can't lock it, don't scan, that's okay. */
|
if (ret != 0)
|
This apparently helps a lot for YCSB, but we need something based on transaction ID visibility for a more general solution: as soon as there are multiple files involved in the checkpoint, this change won't stop O(n^2) behavior.
Author:
{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: Avoid obsolete update checks during long-running transactions. There can potentially be long chains of updates that cannot be freed: there is no point repeatedly scanning them.
refs WT-1913
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/61cd8c419ddf90d181672075040cfe8a8b7367fd
Server:dan:(git)mongo[master]/$ uname -a3.16.0-33-generic #44~14.04.1-Ubuntu24 coreRAID1 SSD for data dirspinning disk for journal124GB RAMmongod (journal on a separate drive):mongod --dbpath /home/dan/wt-data/ --storageEngine wiredTiger --wiredTigerCacheSizeGB 22To load:./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p fieldcount=10 -threads 30To run:./bin/ycsb run mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p operationcount=50000000 -p fieldcount=10 -p requestdistribution=zipfian -threads 90