[SERVER-4820] renaming a collection during bg index build causes bad assertions Created: 31/Jan/12  Updated: 03/Jul/14  Resolved: 29/Nov/12

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Shaun Verch
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-2771 Background index builds on replica se... Closed
Related
related to SERVER-2771 Background index builds on replica se... Closed
Operating System: ALL
Participants:

 Description   

We should also audit for other commands that might not have proper background operation checks.

Test

// Test renaming a collection during a background index build.
 
c = db.jstests_index12;
c.drop();
 
s = startParallelShell (
' for( i = 0; i < 100; ++i ) {' +
'    db.getSisterDB( "admin" ).runCommand( {renameCollection:"test.jstests_index12", to:"test.jstests_index12b"} );' +
'    db.jstests_index12b.drop();' +
'    sleep( 30 );' +
' }'
);
 
for( i = 0; i < 10; ++i ) {
    for( j = 0; j < 1000; ++j ) {
        c.save( {a:j} );
    }
    c.ensureIndex( {i:1}, {background:true} );
    assert( !db.getLastError() );
}
 
s();

Log

Mon Jan 30 22:17:34 [conn2] getFile(): n=65536
Mon Jan 30 22:17:34 [conn2] Assertion: 10295:getFile(): bad file number value (corrupt db?): run repair
0x101a36c15 0x101a71acb 0x101d4b4b4 0x10221b14e 0x10221a77a 0x101c216fb 0x101b058fa 0x101b00327 0x101da41fe 0x101edcd6e 0x101edeba6 0x101edf6b0 0x101ba77b5 0x101ba802a 0x101e52afd 0x101e57819 0x101a21903 0x101e707ba 0x101e71f30 0x101e71f92 
 0   mongod                              0x0000000101a36c15 _ZN5mongo15printStackTraceERSo + 37
 1   mongod                              0x0000000101a71acb _ZN5mongo11msgassertedEiPKc + 363
 2   mongod                              0x0000000101d4b4b4 _ZN5mongo8Database7getFileEiib + 296
 3   mongod                              0x000000010221b14e _ZN5mongo11DataFileMgr9getRecordERKNS_7DiskLocE + 166
 4   mongod                              0x000000010221a77a _ZNK5mongo7DiskLoc3recEv + 24
 5   mongod                              0x0000000101c216fb _ZNK5mongo7DiskLoc3objEv + 43
 6   mongod                              0x0000000101b058fa _ZNK5mongo12IndexDetails14indexNamespaceEv + 58
 7   mongod                              0x0000000101b00327 _ZN5mongo15renameNamespaceEPKcS1_b + 2919
 8   mongod                              0x0000000101da41fe _ZN5mongo19CmdRenameCollection3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 2554
 9   mongod                              0x0000000101edcd6e _ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRNS_14BSONObjBuilderEb + 142
 10  mongod                              0x0000000101edeba6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4678
 11  mongod                              0x0000000101edf6b0 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1472
 12  mongod                              0x0000000101ba77b5 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 117
 13  mongod                              0x0000000101ba802a _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1018
 14  mongod                              0x0000000101e52afd _ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 349
 15  mongod                              0x0000000101e57819 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1049
 16  mongod                              0x0000000101a21903 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 261
 17  mongod                              0x0000000101e707ba _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 1546
 18  mongod                              0x0000000101e71f30 _ZN5boost3_bi5list1INS0_5valueIPN5mongo13MessagingPortEEEEclIPFvS5_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i + 78
 19  mongod                              0x0000000101e71f92 _ZN5boost3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS0_5list1INS0_5valueIS4_EEEEEclEv + 92
Mon Jan 30 22:17:34 [conn1] failed yield
Mon Jan 30 22:17:34 [conn1] ERROR: db gone during bg index?
Mon Jan 30 22:17:34 [conn1] Assertion: 10334:Invalid BSONObj size: 0 (0x00000000) first element: EOO
0x101a36c15 0x101a71acb 0x101a71cea 0x101a5d34c 0x101b8da6c 0x101b05039 0x101c2170f 0x101b057c6 0x101ee0092 0x1021034b9 0x102104dcb 0x102105951 0x101e4ef35 0x101e4f776 0x101e57d52 0x101a21903 0x101e707ba 0x101e71f30 0x101e71f92 0x101e71fbd 
 0   mongod                              0x0000000101a36c15 _ZN5mongo15printStackTraceERSo + 37
 1   mongod                              0x0000000101a71acb _ZN5mongo11msgassertedEiPKc + 363
 2   mongod                              0x0000000101a71cea _ZNK5mongo13ExceptionInfo6appendERNS_14BSONObjBuilderEPKcS4_ + 0
 3   mongod                              0x0000000101a5d34c _ZNK5mongo7BSONObj14_assertInvalidEv + 732
 4   mongod                              0x0000000101b8da6c _ZN5mongo7BSONObj4initEPKc + 76
 5   mongod                              0x0000000101b05039 _ZN5mongo7BSONObjC1EPKNS_6RecordE + 73
 6   mongod                              0x0000000101c2170f _ZNK5mongo7DiskLoc3objEv + 63
 7   mongod                              0x0000000101b057c6 _ZN5mongo16NamespaceDetails15findIndexByNameEPKc + 84
 8   mongod                              0x0000000101ee0092 _ZN5mongo11dropIndexesEPNS_16NamespaceDetailsEPKcS3_RSsRNS_14BSONObjBuilderEb + 914
 9   mongod                              0x00000001021034b9 _ZN5mongo16insert_makeIndexEPNS_16NamespaceDetailsERKSsRKNS_7DiskLocE + 1433
 10  mongod                              0x0000000102104dcb _ZN5mongo11DataFileMgr6insertEPKcPKvibbPb + 4875
 11  mongod                              0x0000000102105951 _ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb + 153
 12  mongod                              0x0000000101e4ef35 _ZN5mongo14checkAndInsertEPKcRNS_7BSONObjE + 229
 13  mongod                              0x0000000101e4f776 _ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE + 1574
 14  mongod                              0x0000000101e57d52 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2386
 15  mongod                              0x0000000101a21903 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 261
 16  mongod                              0x0000000101e707ba _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 1546
 17  mongod                              0x0000000101e71f30 _ZN5boost3_bi5list1INS0_5valueIPN5mongo13MessagingPortEEEEclIPFvS5_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i + 78
 18  mongod                              0x0000000101e71f92 _ZN5boost3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS0_5list1INS0_5valueIS4_EEEEEclEv + 92
 19  mongod                              0x0000000101e71fbd _ZN5boost6detail11thread_dataINS_3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS2_5list1INS2_5valueIS6_EEEEEEE3runEv + 37
Mon Jan 30 22:17:34 [conn1] insert test.system.indexes keyUpdates:0 exception: Invalid BSONObj size: 0 (0x00000000) first element: EOO code:10334 34ms
Mon Jan 30 22:17:34 [conn2] CMD: drop test.jstests_index12b
Mon Jan 30 22:17:34 [conn2] info: assureSysIndexesEmptied cleaned up 1 entries



 Comments   
Comment by Shaun Verch [ 29/Nov/12 ]

This will no longer be an issue once SERVER-2771 is complete since the mechanism for handling collection operations while a background index build is in progress will change.

Comment by Eric Milkie [ 29/Nov/12 ]

We're doing some auditing work in SERVER-2771 to ensure that all commands behave properly during background index builds. This is to support background index builds on secondary nodes.

Comment by auto [ 07/Feb/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-4820 disable test
Branch: master
https://github.com/mongodb/mongo/commit/1578b86d2e6ece9c60b69d8a2db84d9da07d459f

Comment by auto [ 07/Feb/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-4820 test
Branch: master
https://github.com/mongodb/mongo/commit/9d86d5290fa8a13ac803c783311fea03702e6970

Generated at Thu Feb 08 03:07:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.