[SERVER-7426] The usePowerOf2Sizes option corrupts the db permanently and unrecoverably Created: 19/Oct/12  Updated: 07/Mar/14  Resolved: 19/Oct/12

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Mark porter Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Running mongodb 2.2 on Linux version 2.6.32-5-amd64 (Debian 2.6.32-41), with a pymongo driver vintage early 2012.


Issue Links:
Duplicate
duplicates SERVER-7238 power of 2 allocation breaks with doc... Closed
Related
Operating System: ALL
Participants:
Linked BF Score: 0

 Description   

From MongoDB User Google Group:

To improve padding behavior I have added the usePowerOf2Sizes option for a couple of collections:
self.mydb.db.command(

{"collMod" : 'mycoll', 'usePowerOf2Sizes': "true" }

)
A subsequent "update $pushAll" appears to corrupt the database predictably and beyond recovery. The initial error message from the log is appended. With userPowerOf2Sizes commented out, things clunk along happily. Any wisdom on my usage, the nature of the bug, fix availability, etc?

Thu Oct 18 15:12:55 [conn8] update mydb.mycoll query: { pc: 2012696439 } update: { $pushAll: { file_extents: [ [ ObjectId('508050b3fd14054976000024'), [ [ 395, 395 ], [ 403, 403 ], [ 86, 86 ] ] ] ] } } nscanned:657 nupdated:1 fastmodinsert:1 keyUpdates:0 numYields: 5 locks(micros) w:1158 373ms
Thu Oct 18 15:12:59 [conn8] Assertion: 10334:Invalid BSONObj size: 0 (0x00000000) first element: EOO
0xade6e1 0x8036eb 0x80386c 0x571bc7 0xa0e3b5 0xa0ed48 0x5e6081 0xc0862b 0xc0cddd 0xb546af 0xb5dd61 0x56fa52 0x5dbf11 0x7f7720aae8ca 0x7f771fe6192d 
 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xade6e1]
 /usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x9b) [0x8036eb]
 /usr/bin/mongod() [0x80386c]
 /usr/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x497) [0x571bc7]
 /usr/bin/mongod(_ZNK5mongo19CoveredIndexMatcher7matchesERKNS_7BSONObjERKNS_7DiskLocEPNS_12MatchDetailsEb+0x1f5) [0xa0e3b5]
 /usr/bin/mongod(_ZNK5mongo19CoveredIndexMatcher14matchesCurrentEPNS_6CursorEPNS_12MatchDetailsE+0xa8) [0xa0ed48]
 /usr/bin/mongod(_ZN5mongo6Cursor14currentMatchesEPNS_12MatchDetailsE+0x41) [0x5e6081]
 /usr/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyE+0xfeb) [0xc0862b]
 /usr/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0x12d) [0xc0cddd]
 /usr/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0xb546af]
 /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x10b1) [0xb5dd61]
 /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x82) [0x56fa52]
 /usr/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x411) [0x5dbf11]
 /lib/libpthread.so.0(+0x68ca) [0x7f7720aae8ca]
 /lib/libc.so.6(clone+0x6d) [0x7f771fe6192d]
Thu Oct 18 15:12:59 [conn8] update mydb.mycoll query: { pc: 4257797 } update: { $pushAll: { file_extents: [ [ ObjectId('508050b3fd14054976000024'), [ [ 609, 612 ] ] ] ] } } nscanned:766 keyUpdates:0 exception: Invalid BSONObj size: 0 (0x00000000) first element: EOO code:10334 numYields: 5 locks(micros) w:3851 61ms



 Comments   
Comment by Githook User [ 02/Dec/13 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-5625 SERVER-7426 allow all connections in all states to tolerate downed shards and primaries
Branch: v2.4
https://github.com/mongodb/mongo/commit/45662915c2f4e7e19b8333dc93d0d2526d8a34c5

Comment by Mike Cherepov [ 22/Oct/12 ]

Indeed some of the docs in mycoll were busting the 16MB limit after four 5+MB updates; I worked around that, so we are OK for the moment. If your nightly build easily drops into an existing db server installation, I could try that. Otherwise, we could mothball the bug until the next release comes out, whereupon I will test oversized docs and PowerOf2Sizes again.

Comment by Tad Marshall [ 19/Oct/12 ]

Testing a non-corrupt copy of your database with a recent nightly build would show whether you have a new bug, distinct from the case that has been fixed. I understand if that is not feasible. Otherwise, steps-to-reproduce would enable us to test this ourselves.

Comment by Mike Cherepov [ 19/Oct/12 ]

Mark: We have pymongo-2.1.1_-py2.7-linux-x86_64. Recounting the exact steps to reproduce this would be unhelpful without the whole mass of our code, but I would happily supply necessary details. The collection contains docs of a few MB. Here is the snippet that makes a difference between corruption and absence thereof:

  1. self.mycoll.insert( {"dummy":"makenonempty"}

    )

  2. self.mycoll.remove()
  3. self.wtfdb.db.command( {"collMod" : 'mycoll' , 'usePowerOf2Sizes': "true" }

    )

Eliot: Some objects may be getting afoul of the 16MB limit – after removing usePowersOf2Sizes, I saw some Assertions about BSONobj exceeding the limit. Not sure I trust those, though – and didn't this cause mere exceptions pre-2.2.0??. Will investigate the sizes further.

Comment by Mark porter [ 19/Oct/12 ]

Linking to SERVER-7238

Comment by Eliot Horowitz (Inactive) [ 19/Oct/12 ]

SERVER-7238

Generated at Thu Feb 08 03:14:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.