[SERVER-13589] Background index builds with 2.6.0 primary fail to complete on 2.4.x secondary nodes Created: 15/Apr/14  Updated: 11/Jul/16  Resolved: 16/Apr/14

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Replication
Affects Version/s: 2.5.5, 2.6.0
Fix Version/s: 2.6.1, 2.7.0

Type: Bug Priority: Critical - P2
Reporter: Linda Qin Assignee: Eric Milkie
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Operating System: ALL
Backport Completed:
Steps To Reproduce:

1. Start a replica set
2. Insert a document to a collection.

db.docs.insert({x:1})

3. Build an index in background on the primary.

db.docs.ensureIndex( { x: 1 }, { background: true } )

4. It fails to build the index in background on the secondary.

2014-04-15T17:27:31.707+1000 [repl writer worker 1] build index on: test.docs properties: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true }
2014-04-15T17:27:31.708+1000 [repl writer worker 1] index build failed. spec: { v: 1, key: { x: 1.0 }, name: "x_1", ns: "test.docs", background: true } error: 13130 can't start bg index b/c in recursive lock (db.eval?)

Participants:

 Description   
Issue Status as of April 28, 2014

ISSUE SUMMARY
A background index build using the ensureIndex shell helper or the createIndexes command on a 2.6.0 primary replica set member uses the "c" (command) code in the oplog to replicate the index to secondaries. Secondary replica set members fail to build indexes with the new createIndexes command.

USER IMPACT
This bug can lead to inconsistent indexes between primary and secondary members. This may have a serious impact on performance after a replica set fail-over, if queries previously using the index now have to scan the collection. Users affected by this issue should audit their indexes by comparing the system.indexes collections on all replica set members and make sure they are identical. If indexes are missing on the secondaries, please see the documentation page on building indexes on replica sets to manually create them.

WORKAROUNDS
As a workaround to build indexes in the background on a 2.6.0 primary, users can insert a document directly into the system.indexes collection in a database with “background”:true.

Alternatively, starting the mongo shell with the --writeMode=compatibility option avoids the issue altogether.

RESOLUTION
Using the "i" (insert) code on the system.indexes collection for the oplog operation representing the createIndexes command ensures compatibility with older versions of MongoDB and resolves the issue.

AFFECTED VERSIONS
Version 2.6.0 is affected by this bug.

PATCHES
The patch is included in the 2.6.1 production release.

Original description

If you use the new createIndexes() command to build an index in the background, it may not build on replica set secondaries. Inserting an index spec into system.indexes will still work correctly.

Note that by default, the 2.6 MongoDB shell uses the createIndexes() command for the .ensureIndex() shell helper. To force the shell to insert an entry into system.indexes for the .ensureIndex() shell helper, start the shell with the "--writeMode=compatibility" command line parameter.



 Comments   
Comment by Githook User [ 16/Apr/14 ]

Author:

{u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}

Message: SERVER-13589 logOp inserts into system.indexes for index builds

(cherry picked from commit 430cd62d0dfd6d29a9b4768422981c7cd9f1e334)
Branch: v2.6
https://github.com/mongodb/mongo/commit/f70518f8879c1dc3cf7721b6232cc1e84af39182

Comment by Githook User [ 16/Apr/14 ]

Author:

{u'username': u'milkie', u'name': u'Eric Milkie', u'email': u'milkie@10gen.com'}

Message: SERVER-13589 logOp inserts into system.indexes for index builds
Branch: master
https://github.com/mongodb/mongo/commit/430cd62d0dfd6d29a9b4768422981c7cd9f1e334

Generated at Thu Feb 08 03:32:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.