Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1913

Freeing obsolete update lists can be slow during checkpoints

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.6.0
    • Labels:
      None
    • # Replies:
      3
    • Last comment by Customer:
      true

      Description

      At the end of update_serial we check for obsolete update structures. If we can't evict b/c of a checkpoint and that list grows large, this is wasted work inside a transaction.

      Updates shouldn't be blocked, that is, only a single thread at a time is doing the check for obsolete update structures.

      It would be pretty easy to turn off the check for obsolete update structures when checkpoint is running on this file; let's explore ways to tighten it down further using the checkpoint generation and other transactional information.

        Activity

        Hide
        michael.cahill Michael Cahill added a comment -

        Server:
        dan:(git)mongo[master]/$ uname -a
        3.16.0-33-generic #44~14.04.1-Ubuntu 
        24 core
        RAID1 SSD for data dir
        spinning disk for journal
        124GB RAM
         
        mongod (journal on a separate drive):
        mongod --dbpath /home/dan/wt-data/ --storageEngine wiredTiger --wiredTigerCacheSizeGB 22
         
        To load:
        ./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p fieldcount=10 -threads 30
         
        To run:
        ./bin/ycsb run mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p operationcount=50000000 -p fieldcount=10 -p requestdistribution=zipfian -threads 90
        

        Show
        michael.cahill Michael Cahill added a comment - Server: dan:(git)mongo[master]/$ uname -a 3.16.0-33-generic #44~14.04.1-Ubuntu 24 core RAID1 SSD for data dir spinning disk for journal 124GB RAM   mongod (journal on a separate drive): mongod --dbpath /home/dan/wt-data/ --storageEngine wiredTiger --wiredTigerCacheSizeGB 22   To load: ./bin/ycsb load mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p fieldcount=10 -threads 30   To run: ./bin/ycsb run mongodb -s -P workloads/workloada -p mongodb.url=localhost:27017 -p recordcount=20000000 -p operationcount=50000000 -p fieldcount=10 -p requestdistribution=zipfian -threads 90
        Hide
        michael.cahill Michael Cahill added a comment -

        Suggested change:

        diff --git a/src/include/serial.i b/src/include/serial.i
        index e3581ae..afb01b3 100644
        --- a/src/include/serial.i
        +++ b/src/include/serial.i
        @@ -255,7 +255,7 @@ __wt_update_serial(WT_SESSION_IMPL *session, WT_PAGE *page,
                 * obsolete check at a time, and to protect updates from disappearing
                 * under reconciliation.
                 */
        -       if (upd->next != NULL) {
        +       if (!S2BT(session)->checkpointing && upd->next != NULL) {
                        F_CAS_ATOMIC(page, WT_PAGE_SCANNING, ret);
                        /* If we can't lock it, don't scan, that's okay. */
                        if (ret != 0)
        

        This apparently helps a lot for YCSB, but we need something based on transaction ID visibility for a more general solution: as soon as there are multiple files involved in the checkpoint, this change won't stop O(n^2) behavior.

        Show
        michael.cahill Michael Cahill added a comment - Suggested change: diff --git a/src/include/serial.i b/src/include/serial.i index e3581ae..afb01b3 100644 --- a/src/include/serial.i +++ b/src/include/serial.i @@ -255,7 +255,7 @@ __wt_update_serial(WT_SESSION_IMPL *session, WT_PAGE *page, * obsolete check at a time, and to protect updates from disappearing * under reconciliation. */ - if (upd->next != NULL) { + if (!S2BT(session)->checkpointing && upd->next != NULL) { F_CAS_ATOMIC(page, WT_PAGE_SCANNING, ret); /* If we can't lock it, don't scan, that's okay. */ if (ret != 0) This apparently helps a lot for YCSB, but we need something based on transaction ID visibility for a more general solution: as soon as there are multiple files involved in the checkpoint, this change won't stop O(n^2) behavior.
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

        Message: Avoid obsolete update checks during long-running transactions. There can potentially be long chains of updates that cannot be freed: there is no point repeatedly scanning them.

        refs WT-1913
        Branch: develop
        https://github.com/wiredtiger/wiredtiger/commit/61cd8c419ddf90d181672075040cfe8a8b7367fd

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Avoid obsolete update checks during long-running transactions. There can potentially be long chains of updates that cannot be freed: there is no point repeatedly scanning them. refs WT-1913 Branch: develop https://github.com/wiredtiger/wiredtiger/commit/61cd8c419ddf90d181672075040cfe8a8b7367fd

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:
              Days since reply:
              1 year, 51 weeks, 2 days ago
              Date of 1st Reply: