Core Server
  1. Core Server
  2. SERVER-4820

renaming a collection during bg index build causes bad assertions

    Details

    • Type: Bug Bug
    • Status: Closed Closed
    • Priority: Major - P3 Major - P3
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 2.3.2
    • Component/s: Storage
    • Labels:
      None
    • Backport:
      No
    • Operating System:
      ALL
    • Bug Type:
      Unknown
    • # Replies:
      4
    • Last comment by Customer:
      false

      Description

      We should also audit for other commands that might not have proper background operation checks.

      Test

      // Test renaming a collection during a background index build.
      
      c = db.jstests_index12;
      c.drop();
      
      s = startParallelShell (
      ' for( i = 0; i < 100; ++i ) {' +
      '    db.getSisterDB( "admin" ).runCommand( {renameCollection:"test.jstests_index12", to:"test.jstests_index12b"} );' +
      '    db.jstests_index12b.drop();' +
      '    sleep( 30 );' +
      ' }'
      );
      
      for( i = 0; i < 10; ++i ) {
          for( j = 0; j < 1000; ++j ) {
              c.save( {a:j} );
          }
          c.ensureIndex( {i:1}, {background:true} );
          assert( !db.getLastError() );
      }
      
      s();
      

      Log

      Mon Jan 30 22:17:34 [conn2] getFile(): n=65536
      Mon Jan 30 22:17:34 [conn2] Assertion: 10295:getFile(): bad file number value (corrupt db?): run repair
      0x101a36c15 0x101a71acb 0x101d4b4b4 0x10221b14e 0x10221a77a 0x101c216fb 0x101b058fa 0x101b00327 0x101da41fe 0x101edcd6e 0x101edeba6 0x101edf6b0 0x101ba77b5 0x101ba802a 0x101e52afd 0x101e57819 0x101a21903 0x101e707ba 0x101e71f30 0x101e71f92 
       0   mongod                              0x0000000101a36c15 _ZN5mongo15printStackTraceERSo + 37
       1   mongod                              0x0000000101a71acb _ZN5mongo11msgassertedEiPKc + 363
       2   mongod                              0x0000000101d4b4b4 _ZN5mongo8Database7getFileEiib + 296
       3   mongod                              0x000000010221b14e _ZN5mongo11DataFileMgr9getRecordERKNS_7DiskLocE + 166
       4   mongod                              0x000000010221a77a _ZNK5mongo7DiskLoc3recEv + 24
       5   mongod                              0x0000000101c216fb _ZNK5mongo7DiskLoc3objEv + 43
       6   mongod                              0x0000000101b058fa _ZNK5mongo12IndexDetails14indexNamespaceEv + 58
       7   mongod                              0x0000000101b00327 _ZN5mongo15renameNamespaceEPKcS1_b + 2919
       8   mongod                              0x0000000101da41fe _ZN5mongo19CmdRenameCollection3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 2554
       9   mongod                              0x0000000101edcd6e _ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRNS_14BSONObjBuilderEb + 142
       10  mongod                              0x0000000101edeba6 _ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 4678
       11  mongod                              0x0000000101edf6b0 _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1472
       12  mongod                              0x0000000101ba77b5 _ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 117
       13  mongod                              0x0000000101ba802a _ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1018
       14  mongod                              0x0000000101e52afd _ZN5mongoL13receivedQueryERNS_6ClientERNS_10DbResponseERNS_7MessageE + 349
       15  mongod                              0x0000000101e57819 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1049
       16  mongod                              0x0000000101a21903 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 261
       17  mongod                              0x0000000101e707ba _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 1546
       18  mongod                              0x0000000101e71f30 _ZN5boost3_bi5list1INS0_5valueIPN5mongo13MessagingPortEEEEclIPFvS5_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i + 78
       19  mongod                              0x0000000101e71f92 _ZN5boost3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS0_5list1INS0_5valueIS4_EEEEEclEv + 92
      Mon Jan 30 22:17:34 [conn1] failed yield
      Mon Jan 30 22:17:34 [conn1] ERROR: db gone during bg index?
      Mon Jan 30 22:17:34 [conn1] Assertion: 10334:Invalid BSONObj size: 0 (0x00000000) first element: EOO
      0x101a36c15 0x101a71acb 0x101a71cea 0x101a5d34c 0x101b8da6c 0x101b05039 0x101c2170f 0x101b057c6 0x101ee0092 0x1021034b9 0x102104dcb 0x102105951 0x101e4ef35 0x101e4f776 0x101e57d52 0x101a21903 0x101e707ba 0x101e71f30 0x101e71f92 0x101e71fbd 
       0   mongod                              0x0000000101a36c15 _ZN5mongo15printStackTraceERSo + 37
       1   mongod                              0x0000000101a71acb _ZN5mongo11msgassertedEiPKc + 363
       2   mongod                              0x0000000101a71cea _ZNK5mongo13ExceptionInfo6appendERNS_14BSONObjBuilderEPKcS4_ + 0
       3   mongod                              0x0000000101a5d34c _ZNK5mongo7BSONObj14_assertInvalidEv + 732
       4   mongod                              0x0000000101b8da6c _ZN5mongo7BSONObj4initEPKc + 76
       5   mongod                              0x0000000101b05039 _ZN5mongo7BSONObjC1EPKNS_6RecordE + 73
       6   mongod                              0x0000000101c2170f _ZNK5mongo7DiskLoc3objEv + 63
       7   mongod                              0x0000000101b057c6 _ZN5mongo16NamespaceDetails15findIndexByNameEPKc + 84
       8   mongod                              0x0000000101ee0092 _ZN5mongo11dropIndexesEPNS_16NamespaceDetailsEPKcS3_RSsRNS_14BSONObjBuilderEb + 914
       9   mongod                              0x00000001021034b9 _ZN5mongo16insert_makeIndexEPNS_16NamespaceDetailsERKSsRKNS_7DiskLocE + 1433
       10  mongod                              0x0000000102104dcb _ZN5mongo11DataFileMgr6insertEPKcPKvibbPb + 4875
       11  mongod                              0x0000000102105951 _ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb + 153
       12  mongod                              0x0000000101e4ef35 _ZN5mongo14checkAndInsertEPKcRNS_7BSONObjE + 229
       13  mongod                              0x0000000101e4f776 _ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE + 1574
       14  mongod                              0x0000000101e57d52 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 2386
       15  mongod                              0x0000000101a21903 _ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE + 261
       16  mongod                              0x0000000101e707ba _ZN5mongo3pms9threadRunEPNS_13MessagingPortE + 1546
       17  mongod                              0x0000000101e71f30 _ZN5boost3_bi5list1INS0_5valueIPN5mongo13MessagingPortEEEEclIPFvS5_ENS0_5list0EEEvNS0_4typeIvEERT_RT0_i + 78
       18  mongod                              0x0000000101e71f92 _ZN5boost3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS0_5list1INS0_5valueIS4_EEEEEclEv + 92
       19  mongod                              0x0000000101e71fbd _ZN5boost6detail11thread_dataINS_3_bi6bind_tIvPFvPN5mongo13MessagingPortEENS2_5list1INS2_5valueIS6_EEEEEEE3runEv + 37
      Mon Jan 30 22:17:34 [conn1] insert test.system.indexes keyUpdates:0 exception: Invalid BSONObj size: 0 (0x00000000) first element: EOO code:10334 34ms
      Mon Jan 30 22:17:34 [conn2] CMD: drop test.jstests_index12b
      Mon Jan 30 22:17:34 [conn2] info: assureSysIndexesEmptied cleaned up 1 entries
      

        Issue Links

          Activity

          Hide
          auto
          added a comment -

          Author:

          {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

          Message: SERVER-4820 test
          Branch: master
          https://github.com/mongodb/mongo/commit/9d86d5290fa8a13ac803c783311fea03702e6970

          Show
          auto
          added a comment - Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'} Message: SERVER-4820 test Branch: master https://github.com/mongodb/mongo/commit/9d86d5290fa8a13ac803c783311fea03702e6970
          Hide
          auto
          added a comment -

          Author:

          {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

          Message: SERVER-4820 disable test
          Branch: master
          https://github.com/mongodb/mongo/commit/1578b86d2e6ece9c60b69d8a2db84d9da07d459f

          Show
          auto
          added a comment - Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'} Message: SERVER-4820 disable test Branch: master https://github.com/mongodb/mongo/commit/1578b86d2e6ece9c60b69d8a2db84d9da07d459f
          Hide
          Eric Milkie
          added a comment -

          We're doing some auditing work in SERVER-2771 to ensure that all commands behave properly during background index builds. This is to support background index builds on secondary nodes.

          Show
          Eric Milkie
          added a comment - We're doing some auditing work in SERVER-2771 to ensure that all commands behave properly during background index builds. This is to support background index builds on secondary nodes.
          Hide
          Shaun Verch
          added a comment -

          This will no longer be an issue once SERVER-2771 is complete since the mechanism for handling collection operations while a background index build is in progress will change.

          Show
          Shaun Verch
          added a comment - This will no longer be an issue once SERVER-2771 is complete since the mechanism for handling collection operations while a background index build is in progress will change.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 21 weeks ago
                Date of 1st Reply: