Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-14186

rs.stepDown during mapReduce causes fassert in logOp

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • 2.6.2, 2.7.4
    • Affects Version/s: 2.6.1
    • Component/s: MapReduce, Replication
    • None
    • Fully Compatible
    • ALL

      Issue Status as of Jun 10, 2014

      ISSUE SUMMARY
      When a server in a replica set steps down from its primary role while certain metadata commands are actively running, the server may shut down. Such commands include mapReduce, renameCollection, dropDatabase, drop, dropIndexes, emptyCapped, convertToCapped, collMod, and create.

      USER IMPACT
      Replica sets with a sufficient majority should continue to operate normally after a new primary is elected, but any metadata write operations in progress are lost and not replicated. The affected server can be restarted as a secondary.

      WORKAROUNDS
      N/A

      AFFECTED VERSIONS
      MongoDB 2.6.0 and 2.6.1 production releases are affected by this issue.

      FIX VERSION
      The fix is included in the 2.6.2 production release.

      RESOLUTION DETAILS
      Apply proper locking semantics around metacommands to synchronize with replica set stepdown.

      Original description

      Create a replica set, run mapReduce's in a tight loop:

      function mr() {
          db.c.drop()
          db.c.insert({})
          while (true) {
              map = function() {}
              reduce = function() {}
              db.c.mapReduce(map, reduce, {out:'out'})
          }
      }
      

      Then do an rs.stepDown(); produces fassert in logOp:

      2014-06-06T10:51:24.741-0400 [conn21] replSet error : logOp() but not primary
      2014-06-06T10:51:24.742-0400 [conn21] test Fatal Assertion 17405
      2014-06-06T10:51:24.752-0400 [conn21] test 0x1006ac7ab 0x1006642c2 0x100653e3e 0x1004692e8 0x10046cb0f 0x1001b83b9 0x1001b907c 0x1003cfc7f 0x1002a1050 0x100297b3f 0x100297e50 0x1000550d4 0x1000355ac 0x100297661 0x10003e802 0x1000318a6 0x100031fdf 0x1002a3c04 0x1001552c5 0x10015b854 
       0   mongod                              0x00000001006ac7ab _ZN5mongo15printStackTraceERSo + 43
       1   mongod                              0x00000001006642c2 _ZN5mongo10logContextEPKc + 114
       2   mongod                              0x0000000100653e3e _ZN5mongo13fassertFailedEi + 110
       3   mongod                              0x00000001004692e8 _ZN5mongoL8_logOpRSEPKcS1_S1_RKNS_7BSONObjEPS2_Pbb + 376
       4   mongod                              0x000000010046cb0f _ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_PbbPS3_ + 79
       5   mongod                              0x00000001001b83b9 _ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb + 2941
       6   mongod                              0x00000001001b907c _ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi + 1388
       7   mongod                              0x00000001003cfc7f _ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_ + 1615
       8   mongod                              0x00000001002a1050 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE + 1968
       9   mongod                              0x0000000100297b3f _ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs + 145
       10  mongod                              0x0000000100297e50 _ZThn136_N5mongo14DBDirectClient4callERNS_7MessageES2_bPSs + 16
       11  mongod                              0x00000001000550d4 _ZN5mongo14DBClientCursor4initEv + 346
       12  mongod                              0x00000001000355ac _ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 180
       13  mongod                              0x0000000100297661 _ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 79
       14  mongod                              0x000000010003e802 _ZN5mongo17DBClientInterface5findNERSt6vectorINS_7BSONObjESaIS2_EERKSsNS_5QueryEiiPKS2_i + 156
       15  mongod                              0x00000001000318a6 _ZN5mongo17DBClientInterface7findOneERKSsRKNS_5QueryEPKNS_7BSONObjEi + 118
       16  mongod                              0x0000000100031fdf _ZN5mongo20DBClientWithCommands10runCommandERKSsRKNS_7BSONObjERS3_i + 891
       17  mongod                              0x00000001002a3c04 _ZN5mongo20DBClientWithCommands14dropCollectionERKSsPNS_7BSONObjE + 380
       18  mongod                              0x00000001001552c5 _ZN5mongo2mr5State19dropTempCollectionsEv + 75
       19  mongod                              0x000000010015b854 _ZN5mongo2mr16MapReduceCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb + 5468
      

            Assignee:
            kaloian.manassiev@mongodb.com Kaloian Manassiev
            Reporter:
            bruce.lucas@mongodb.com Bruce Lucas (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved: