Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.7.0
    • Labels:
      None
    • # Replies:
      15
    • Last comment by Customer:
      true

      Description

      The mongodb test jstests/noPassthroughWithMongod/indexbg_interrupts.js takes significantly longer with snapshots enabled (42 min vs 1 min). Most of the time is spent in the convertToCapped command. Profiling shows that the vast majority of the time is spent iterating the update lists in __rec_txn_read and __wt_row_modify.

      My theory is that the existence of a snapshot is preventing splitting a page. This workload creates a new collection and inserts ~100k records into it without creating new snapshots or retiring old ones. I think that the linear scans are causing this to be an O(N^2) operation.

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: WT-2042 Only try to evict tombstones that are visible to all readers.

          Merge pull request #2196 from wiredtiger/WT-2042

          (cherry picked from commit 5ab26af636f8c6a5a86200d1c18f249d0749fc9a)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/08e884609d9be6bcff8bb580e3ce85ef2b5a6b5d

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: WT-2042 Only try to evict tombstones that are visible to all readers. Merge pull request #2196 from wiredtiger/ WT-2042 (cherry picked from commit 5ab26af636f8c6a5a86200d1c18f249d0749fc9a) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/08e884609d9be6bcff8bb580e3ce85ef2b5a6b5d
          Hide
          michael.cahill Michael Cahill added a comment -

          Note that the changes so far on the WiredTiger side improve the situation, but there is still inefficiency at the beginning of the tree where lots of tombstones accumulate. Changing MongoDB to track the first live record in the tree for capped delete operations removes the remaining inefficiency, but we will investigate a more general approach in WiredTiger first.

          Show
          michael.cahill Michael Cahill added a comment - Note that the changes so far on the WiredTiger side improve the situation, but there is still inefficiency at the beginning of the tree where lots of tombstones accumulate. Changing MongoDB to track the first live record in the tree for capped delete operations removes the remaining inefficiency, but we will investigate a more general approach in WiredTiger first.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

          Message: SERVER-19213 Disable convertToCapped testing in indexbg_interrupts.js

          It is way too slow with the replSnapshotThread enabled due to WT-2042. It
          will be re-enabled as part of SERVER-19910.
          Branch: master
          https://github.com/mongodb/mongo/commit/1abd9f24692b8fc95767f2575594fb232cba2157

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'} Message: SERVER-19213 Disable convertToCapped testing in indexbg_interrupts.js It is way too slow with the replSnapshotThread enabled due to WT-2042 . It will be re-enabled as part of SERVER-19910 . Branch: master https://github.com/mongodb/mongo/commit/1abd9f24692b8fc95767f2575594fb232cba2157
          Hide
          pasette Dan Pasette added a comment -

          Michael, seems like this issue can be resolved as the WT portion of the work is complete, and the rest will be done in the integration layer with SERVER-21008.

          Show
          pasette Dan Pasette added a comment - Michael, seems like this issue can be resolved as the WT portion of the work is complete, and the rest will be done in the integration layer with SERVER-21008 .
          Hide
          michael.cahill Michael Cahill added a comment -

          Agreed, I'll resolve.

          Show
          michael.cahill Michael Cahill added a comment - Agreed, I'll resolve.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 26 weeks, 6 days ago
                Date of 1st Reply: