[SERVER-19513] Truncating a capped collection may not unindex deleted documents in WiredTiger Created: 21/Jul/15  Updated: 19/Sep/15  Resolved: 22/Jul/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.0
Fix Version/s: 3.0.5, 3.1.7

Type: Bug Priority: Critical - P2
Reporter: Geert Bosch Assignee: Geert Bosch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Steps To Reproduce:

/**
 * Test 'captrunc' command on indexed capped collections0
 */
(function() {
    'use strict';
 
    db.cappedb.drop();
    db._dbCommand({ create: "cappedb", capped: true, size: 1000, autoIndexId: true });
    var t = db.cappedb;
 
    for (var j = 1; j < 10; j++) {
        assert.eq(t.insert({x:j}),{nInserted:1});
    }
 
    assert.commandWorked(db.runCommand({ captrunc: "cappedb", n: 5, inc: false }));
    var last = t.find({},{_id:1}).sort({_id:-1}).next();
    assert.neq(null, t.findOne({_id: last._id}), 
               tojson(last) + " is in _id index, but not in capped collection");
})();

Sprint: Quint Iteration 7
Participants:

 Description   
Issue Status as of Jul 22, 2015

ISSUE SUMMARY
Truncating a capped collection using the WiredTiger storage engine may not remove the deleted documents from the indexes in the collection.

USER IMPACT
When running the captrunc command on a capped collection, or when a member of a replica-set does a rollback of such a collection, the removed documents may not be deleted from the collection's indexes, thus leaving the indexes in an inconsistent state. As a result, a query covered by an index may report results that include deleted documents, and queries using an index in general may fail or return incorrect results. Regular deletions as result of exceeding capped collection capacity are not affected by this issue.

To determine if a capped collection is impacted by this issue, users can compare the output of the following commands:

  • db.cappedcollection.distinct("_id").length
  • db.cappedcollection.count()

If the results are equal the capped collection is not impacted by this issue. Alternatively, users can compare the nrecords and keysPerIndex fields of the db.cappedcollection.validate() output in the mongo shell; if the values are equal then the capped collection is not impacted.

WORKAROUNDS
Users may run the reIndex command on the affected capped collections. This restores consistency in the collection's indexes, but does not prevent the issue from happening again.

AFFECTED VERSIONS
MongoDB versions 3.0.0 through 3.0.4 using the WiredTiger storage engine.

FIX VERSION
The fix is included in the 3.0.5 production release.

Original description

The captrunc command does not unindex documents removed from a WiredTiger capped collection. As rollback operations in a replicaset are performed using this command, they can leave indexes in an inconsistent state.



 Comments   
Comment by Githook User [ 28/Jul/15 ]

Author:

{u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

Message: SERVER-19513 move functional test to noPassthroughWithMongod
Branch: master
https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848

Comment by Githook User [ 28/Jul/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-19513: Unindex documents when truncating WT capped collections
Branch: master
https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

Message: SERVER-19513 move functional test to noPassthroughWithMongod

(cherry picked from commit 162fb82bed75ede539bbb960f44d232f8b8e5848)
Branch: v3.0
https://github.com/mongodb/mongo/commit/c1c1e095ba0b813b811d8eb2587b42583192c030

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@10gen.com'}

Message: SERVER-19513 move functional test to noPassthroughWithMongod
Branch: master
https://github.com/mongodb/mongo/commit/162fb82bed75ede539bbb960f44d232f8b8e5848

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'}

Message: Revert "Revert "SERVER-19513: Unindex documents when truncating WT capped collections""

This reverts commit afbf47d2dabcdd1c68b5acc065544aec858a9d66.
Branch: v3.0
https://github.com/mongodb/mongo/commit/4829a881dd5eb4f86ff6ba38c29d3abd652481a3

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'ramonfm', u'name': u'Ramon Fernandez', u'email': u'ramon.fernandez@mongodb.com'}

Message: Revert "SERVER-19513: Unindex documents when truncating WT capped collections"

This reverts commit f40286731678938e1f80653473743de8742120ab.
Branch: v3.0
https://github.com/mongodb/mongo/commit/afbf47d2dabcdd1c68b5acc065544aec858a9d66

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-19513: Unindex documents when truncating WT capped collections

(cherry-picked from f6a55c1a9695f67b73e1ead4a1845ac95daef983)

Conflicts:
src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp
Branch: v3.0
https://github.com/mongodb/mongo/commit/f40286731678938e1f80653473743de8742120ab

Comment by Githook User [ 22/Jul/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-19513: Unindex documents when truncating WT capped collections
Branch: master
https://github.com/mongodb/mongo/commit/f6a55c1a9695f67b73e1ead4a1845ac95daef983

Generated at Thu Feb 08 03:51:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.