[SERVER-6981] maxPasses assertion (on allocation failure) can make capped collection unreadable Created: 09/Sep/12  Updated: 03/Dec/18  Resolved: 08/Dec/14

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 2.8.0-rc3

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Mathias Stearn
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-15265 passes >= maxPasses (capped collection) Closed
is duplicated by SERVER-15303 passes >= maxPasses in oplog Closed
is duplicated by SERVER-8666 adding large docs to capped collectio... Closed
Related
related to SERVER-13914 'no space in capped collection' error... Closed
related to SERVER-12058 Primary should abort if encountered p... Closed
is related to SERVER-17561 Fragmentation/non-linearity in mmapv1... Closed
Tested
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

The maxPasses assertion (triggered on repeated allocation failure from a capped collection) can leave capped collection in non iterable state. In particular

  • the capExtent extent can be empty (contain no documents)
  • it's possible to have capExtent non null and capFirstNewRecord null, in which case the capped cursors will not work properly

Should also investigate when the maxPasses assertion can be triggered.

            if( ++passes > maxPasses ) {
                log() << "passes ns:" << ns << " len:" << len << " maxPasses: " << maxPasses << '\n';
                log() << "passes max:" << maxCappedDocs() << " nrecords:" << stats.nrecords << " datasize: " << stats.datasize << endl;
                massert( 10345 ,  "passes >= maxPasses in capped collection alloc", false );
            }



 Comments   
Comment by David Ffrench [ 03/Dec/18 ]

My apologies Eric and thank you for the quick response. I have created a post on the mongodb-user group here. https://groups.google.com/forum/#!topic/mongodb-user/bABNI7te1J0

 

Comment by Eric Milkie [ 28/Nov/18 ]

Hi David, I don't think your question is related to this ticket. The SERVER project in general is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-user group. Thank you!

Comment by David Ffrench [ 28/Nov/18 ]

Apologies, I know this issue hasn't been updated in quite some time but I am hoping someone will be able to provide an answer for me.

In a scenario with a replica set, do you know if having a write concern > 1 will ensure the operation has been successful on the secondaries?

Comment by James Blackburn [ 02/Sep/15 ]

I think this data-loss bug can affect the operation log, which means that MongoDB will randomly fail to replicate documents to secondaries. However it successfully writes the document to the primary! It appears this would particularly affect the oplog due to the unconstrained nature of the write workload to this collection.

As far as I can see this issue exists in the latest 2.4.x and 2.6.x stable branches:
https://github.com/mongodb/mongo/blob/v2.4/src/mongo/db/cap.cpp#L276
https://github.com/mongodb/mongo/blob/v2.6/src/mongo/db/structure/catalog/cap.cpp#L295

This is likely the root cause of the sporadic, and persistent, data loss bugs we've encountered in 2.2.x and 2.4.x CS-23187 CS-8955

Comment by Githook User [ 08/Dec/14 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-6981 Remove maxPasses limit and unify capped no-space errors
Branch: master
https://github.com/mongodb/mongo/commit/545b4f727d2b74f1a50034cf63f22911a2dd0962

Generated at Thu Feb 08 03:13:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.