[SERVER-12312] getIndexes() on mongos reads from wrong shard if read preference is set Created: 10/Jan/14  Updated: 11/Jul/16  Resolved: 05/Feb/14

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.4.8, 2.5.4
Fix Version/s: 2.6.0-rc0

Type: Bug Priority: Major - P3
Reporter: Thomas Rueckstiess Assignee: Randolph Tan
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

the second jstest (sh_index2.js) also triggers the bug on 2.5.4.


Attachments: File sh_index.js     File sh_index2.js    
Issue Links:
Depends
Operating System: ALL
Participants:

 Description   

For a sharded collection, a call to .getIndexes() or querying the system.indexes collection with a namespace specified (those are equivalent) on a mongos will be directed to the first shard that has chunks of that collection, NOT to the primary shard. This is due to a deliberate hack, see:
https://github.com/mongodb/mongo/blob/1b2af152b9806b0b3443bd4facddf87c8adecb0e/src/mongo/s/strategy.cpp#L95

However, this hack is being circumvented if a read preference is set on the query or if the query has an additional sort. (Other conditions may trigger the same behavior as well, those are the ones we have found so far).

This seems to be a bug in the kernel, as the read preference has nothing to do with what shard is being selected.

In particular, some drivers (Node.js for example) always set a default read preference of primary, therefore circumventing the "hack" and reading the indexes always from the primary shard, not from the first chunk-containing shard.

This is especially problematic if an index is created through mongos where the collection has no chunks on the primary shard (due to tagging for example). The index is then not visible when queried with read preference.

jstests to reproduce behavior (two variants) attached.



 Comments   
Comment by Githook User [ 05/Feb/14 ]

Author:

{u'username': u'renctan', u'name': u'Randolph Tan', u'email': u'randolph@10gen.com'}

Message: SERVER-12312 getIndexes() on mongos reads from wrong shard if read preference is set
Branch: master
https://github.com/mongodb/mongo/commit/86e948fdf8e900fa187f9b34d85841bca580729a

Comment by Thomas Rueckstiess [ 10/Jan/14 ]

Attaching a second jstest sh_index2.js that shows a more realistic (and more severe) consequence of this bug.

  1. build index from mongos on a sharded collection that does not have chunks on the primary shard
  2. read the indexes back, with a read preference

The index appears to not exist, because it wasn't built on the primary shard (no chunks on it) but the indexes were read from the primary shard.

mongos> db.system.indexes.find({ns: "test.docs"})
{ "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.docs", "name" : "_id_" }
{ "v" : 1, "key" : { "foo" : 1 }, "ns" : "test.docs", "name" : "foo_1" }
mongos> db.docs.ensureIndex({new_index: 1})
mongos> db.system.indexes.find({ns: "test.docs"}).readPref('primary')
{ "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.docs", "name" : "_id_" }
{ "v" : 1, "key" : { "foo" : 1 }, "ns" : "test.docs", "name" : "foo_1" }
mongos> db.system.indexes.find({ns: "test.docs"})
{ "v" : 1, "key" : { "_id" : 1 }, "ns" : "test.docs", "name" : "_id_" }
{ "v" : 1, "key" : { "foo" : 1 }, "ns" : "test.docs", "name" : "foo_1" }
{ "v" : 1, "key" : { "new_index" : 1 }, "ns" : "test.docs", "name" : "new_index_1" }

Generated at Thu Feb 08 03:28:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.