[SERVER-13404] 2.6 secondaries abort when replicated index spec contains a dollar-sign field Created: 28/Mar/14  Updated: 08/Dec/15  Resolved: 08/Dec/15

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 2.6.0-rc2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kamran K. Assignee: Unassigned
Resolution: Won't Fix Votes: 0
Labels: 26qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-13496 Creating index with same name but dif... Closed
is related to SERVER-13206 Mixed-version 2.4/2.6 replica set cra... Closed
is related to SERVER-12834 Create flag to allow mongod to ignore... Closed
Operating System: ALL
Steps To Reproduce:

var replSet = new ReplSetTest({
    name: 'mixedVersionIndexes',
    nodes: 3,
    nodeOptions: {
        smallfiles: '',
        binVersion: MongoRunner.versionIterator(['2.4', '2.6'])
    }
});
 
replSet.startSet();
replSet.initiate();
replSet.awaitSecondaryNodes();
 
// make a 2.4 node the primary
assert.soon(function() {
    var primary = replSet.getPrimary();
 
    if (primary.getDB('admin').version().startsWith('2.4') === false) {
        try {
            primary.getDB('admin').runCommand({replSetStepDown: 60});
        } catch(e) {
            jsTest.log(e);
        }
        return false;
    }
 
    return true;
}, 'failed to elect 2.4 node as primary', 60000, 30000);
 
// create an invalid 2.6 index on the 2.4 primary
var coll = replSet.getPrimary().getDB('foo').bar;
coll.ensureIndex({$foo:1});
 
replSet.awaitReplication();
 
jsTest.log('available nodes:\n' + tojson(replSet.liveNodes));

Participants:

 Description   

A 2.6 secondary will abort if a replicated index spec contains a field with a dollar sign. This bug is similar to SERVER-13206, which had a fairly narrow fix and may be missing other CannotCreateIndex scenarios.

Output on secondaries:

 m31000| 2014-03-28T15:20:27.384-0400 [repl writer worker 1] ERROR: writer worker caught exception:  :: caused by :: 67 bad index key pattern { $foo: 1.0 }: Index key contains an illegal field name: field name starts with '$'. on: { ts: Timestamp 1396034426000|1, h: -8749786978335064046, v: 2, op: "i", ns: "foo.system.indexes", o: { _id: ObjectId('5335cb7ad7a2aa0c289a8882'), ns: "foo.bar", key: { $foo: 1.0 }, name: "$foo_1" } }
 m31000| 2014-03-28T15:20:27.384-0400 [repl writer worker 1] Fatal Assertion 16360
 m31000| 2014-03-28T15:20:27.384-0400 [repl writer worker 1] 
 m31000| 
 m31000| ***aborting after fassert() failure
 m31000| 


Version: 6412fa7eca5c834f29490ab6cdd1cb13b3d99c97



 Comments   
Comment by J Rassi [ 18/Apr/14 ]

I'm starting to think it's correct to let 2.6 secondaries halt replication if they encounter these indexes (the defense being that 2.6 just does not support them).

Though I do think that it's worth it to consider adding an "index pre-validation" step to initial sync, so that the initial sync can fail fast if it would otherwise replicate an unsupported index.

Comment by J Rassi [ 02/Apr/14 ]

Note this also affects initial sync, where the sync source has a "dollar index" or "empty string index".

I think skipping index key pattern validation on secondaries (in style of SERVER-13206) is a candidate fix worth considering.

We could additionally pre-validate indexes at the start of initial sync (in style of SERVER-12242), but that wouldn't help the normal replication path.

Comment by Eric Milkie [ 28/Mar/14 ]

Actually, the code that took care of this was enhanced in SERVER-12834. Nevertheless, should still do something for $ fields in indexes.

Comment by Eric Milkie [ 28/Mar/14 ]

Yes, that was fixed in SERVER-12834. We should consider doing a similar fix for this situation.

Comment by Kamran K. [ 28/Mar/14 ]

When I insert an unindexable document on a 2.4 primary, I see this on the 2.6 secondaries:

 m31002| 2014-03-28T15:51:50.070-0400 [repl writer worker 1] foo.bar Btree::insert: key too large to index, failing foo.bar.$foo_1 10012 { : "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa..." }

I don't see any aborts though.

Comment by Eric Milkie [ 28/Mar/14 ]

I think this is the same problem as if you insert a document on 2.4 that is unindexable due to the key length restriction. 2.4 allows this and 2.6 does not.

Generated at Thu Feb 08 03:31:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.