Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13589

Background index builds with 2.6.0 primary fail to complete on 2.4.x secondary nodes

XMLWordPrintableJSON

    • ALL
    • Hide

      1. Start a replica set
      2. Insert a document to a collection.

      db.docs.insert({x:1})

      3. Build an index in background on the primary.

      db.docs.ensureIndex( { x: 1 }, { background: true } )

      4. It fails to build the index in background on the secondary.

      2014-04-15T17:27:31.707+1000 [repl writer worker 1] build index on: test.docs properties: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true }
2014-04-15T17:27:31.708+1000 [repl writer worker 1] index build failed. spec: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true } error: 13130 can't start bg index b/c in recursive lock (db.eval?)
      Show
      1. Start a replica set 2. Insert a document to a collection. db.docs.insert({x:1}) 3. Build an index in background on the primary. db.docs.ensureIndex( { x: 1 }, { background: true } ) 4. It fails to build the index in background on the secondary. 2014-04-15T17:27:31.707+1000 [repl writer worker 1] build index on: test.docs properties: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true } 2014-04-15T17:27:31.708+1000 [repl writer worker 1] index build failed. spec: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true } error: 13130 can't start bg index b/c in recursive lock (db.eval?)

      Issue Status as of April 28, 2014

      ISSUE SUMMARY
      A background index build using the ensureIndex shell helper or the createIndexes command on a 2.6.0 primary replica set member uses the "c" (command) code in the oplog to replicate the index to secondaries. Secondary replica set members fail to build indexes with the new createIndexes command.

      USER IMPACT
      This bug can lead to inconsistent indexes between primary and secondary members. This may have a serious impact on performance after a replica set fail-over, if queries previously using the index now have to scan the collection. Users affected by this issue should audit their indexes by comparing the system.indexes collections on all replica set members and make sure they are identical. If indexes are missing on the secondaries, please see the documentation page on building indexes on replica sets to manually create them.

      WORKAROUNDS
      As a workaround to build indexes in the background on a 2.6.0 primary, users can insert a document directly into the system.indexes collection in a database with “background”:true.

      Alternatively, starting the mongo shell with the --writeMode=compatibility option avoids the issue altogether.

      RESOLUTION
      Using the "i" (insert) code on the system.indexes collection for the oplog operation representing the createIndexes command ensures compatibility with older versions of MongoDB and resolves the issue.

      AFFECTED VERSIONS
      Version 2.6.0 is affected by this bug.

      PATCHES
      The patch is included in the 2.6.1 production release.

      Original description

      If you use the new createIndexes() command to build an index in the background, it may not build on replica set secondaries. Inserting an index spec into system.indexes will still work correctly.

      Note that by default, the 2.6 MongoDB shell uses the createIndexes() command for the .ensureIndex() shell helper. To force the shell to insert an entry into system.indexes for the .ensureIndex() shell helper, start the shell with the "--writeMode=compatibility" command line parameter.

            Assignee:
            milkie@mongodb.com Eric Milkie
            Reporter:
            linda.qin@mongodb.com Linda Qin
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: