Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19513

Truncating a capped collection may not unindex deleted documents in WiredTiger

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.5, 3.1.7
    • Component/s: WiredTiger
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      /**
       * Test 'captrunc' command on indexed capped collections0
       */
      (function() {
          'use strict';
       
          db.cappedb.drop();
          db._dbCommand({ create: "cappedb", capped: true, size: 1000, autoIndexId: true });
          var t = db.cappedb;
       
          for (var j = 1; j < 10; j++) {
              assert.eq(t.insert({x:j}),{nInserted:1});
          }
       
          assert.commandWorked(db.runCommand({ captrunc: "cappedb", n: 5, inc: false }));
          var last = t.find({},{_id:1}).sort({_id:-1}).next();
          assert.neq(null, t.findOne({_id: last._id}), 
                     tojson(last) + " is in _id index, but not in capped collection");
      })();
      

      Show
      /** * Test 'captrunc' command on indexed capped collections0 */ (function() { 'use strict';   db.cappedb.drop(); db._dbCommand({ create: "cappedb", capped: true, size: 1000, autoIndexId: true }); var t = db.cappedb;   for (var j = 1; j < 10; j++) { assert.eq(t.insert({x:j}),{nInserted:1}); }   assert.commandWorked(db.runCommand({ captrunc: "cappedb", n: 5, inc: false })); var last = t.find({},{_id:1}).sort({_id:-1}).next(); assert.neq(null, t.findOne({_id: last._id}), tojson(last) + " is in _id index, but not in capped collection"); })();
    • Sprint:
      Quint Iteration 7

      Description

      Issue Status as of Jul 22, 2015

      ISSUE SUMMARY
      Truncating a capped collection using the WiredTiger storage engine may not remove the deleted documents from the indexes in the collection.

      USER IMPACT
      When running the captrunc command on a capped collection, or when a member of a replica-set does a rollback of such a collection, the removed documents may not be deleted from the collection's indexes, thus leaving the indexes in an inconsistent state. As a result, a query covered by an index may report results that include deleted documents, and queries using an index in general may fail or return incorrect results. Regular deletions as result of exceeding capped collection capacity are not affected by this issue.

      To determine if a capped collection is impacted by this issue, users can compare the output of the following commands:

      • db.cappedcollection.distinct("_id").length
      • db.cappedcollection.count()

      If the results are equal the capped collection is not impacted by this issue. Alternatively, users can compare the nrecords and keysPerIndex fields of the db.cappedcollection.validate() output in the mongo shell; if the values are equal then the capped collection is not impacted.

      WORKAROUNDS
      Users may run the reIndex command on the affected capped collections. This restores consistency in the collection's indexes, but does not prevent the issue from happening again.

      AFFECTED VERSIONS
      MongoDB versions 3.0.0 through 3.0.4 using the WiredTiger storage engine.

      FIX VERSION
      The fix is included in the 3.0.5 production release.

      Original description

      The captrunc command does not unindex documents removed from a WiredTiger capped collection. As rollback operations in a replicaset are performed using this command, they can leave indexes in an inconsistent state.

        Activity

        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

        Message: SERVER-19513: Unindex documents when truncating WT capped collections
        Branch: master
        https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-19513 : Unindex documents when truncating WT capped collections Branch: master https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

        Message: SERVER-19513: Unindex documents when truncating WT capped collections

        (cherry-picked from f6a55c1a9695f67b73e1ead4a1845ac95daef983)

        Conflicts:
        src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/f40286731678938e1f80653473743de8742120ab

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-19513 : Unindex documents when truncating WT capped collections (cherry-picked from f6a55c1a9695f67b73e1ead4a1845ac95daef983) Conflicts: src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp Branch: v3.0 https://github.com/mongodb/mongo/commit/f40286731678938e1f80653473743de8742120ab
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'}

        Message: Revert "SERVER-19513: Unindex documents when truncating WT capped collections"

        This reverts commit f40286731678938e1f80653473743de8742120ab.
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/afbf47d2dabcdd1c68b5acc065544aec858a9d66

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'} Message: Revert " SERVER-19513 : Unindex documents when truncating WT capped collections" This reverts commit f40286731678938e1f80653473743de8742120ab. Branch: v3.0 https://github.com/mongodb/mongo/commit/afbf47d2dabcdd1c68b5acc065544aec858a9d66
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'}

        Message: Revert "Revert "SERVER-19513: Unindex documents when truncating WT capped collections""

        This reverts commit afbf47d2dabcdd1c68b5acc065544aec858a9d66.
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/4829a881dd5eb4f86ff6ba38c29d3abd652481a3

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'} Message: Revert "Revert " SERVER-19513 : Unindex documents when truncating WT capped collections"" This reverts commit afbf47d2dabcdd1c68b5acc065544aec858a9d66. Branch: v3.0 https://github.com/mongodb/mongo/commit/4829a881dd5eb4f86ff6ba38c29d3abd652481a3
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

        Message: SERVER-19513 move functional test to noPassthroughWithMongod
        Branch: master
        https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'} Message: SERVER-19513 move functional test to noPassthroughWithMongod Branch: master https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

        Message: SERVER-19513 move functional test to noPassthroughWithMongod

        (cherry picked from commit 162fb82bed75ede539bbb960f44d232f8b8e5848)
        Branch: v3.0
        https://github.com/mongodb/mongo/commit/c1c1e095ba0b813b811d8eb2587b42583192c030

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'} Message: SERVER-19513 move functional test to noPassthroughWithMongod (cherry picked from commit 162fb82bed75ede539bbb960f44d232f8b8e5848) Branch: v3.0 https://github.com/mongodb/mongo/commit/c1c1e095ba0b813b811d8eb2587b42583192c030
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

        Message: SERVER-19513: Unindex documents when truncating WT capped collections
        Branch: master
        https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-19513 : Unindex documents when truncating WT capped collections Branch: master https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983
        Hide
        xgen-internal-githook Githook User added a comment -

        Author:

        {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

        Message: SERVER-19513 move functional test to noPassthroughWithMongod
        Branch: master
        https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848

        Show
        xgen-internal-githook Githook User added a comment - Author: {u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'} Message: SERVER-19513 move functional test to noPassthroughWithMongod Branch: master https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

                Agile