[SERVER-31511] AllowPartialResults not working when sharded cluster is backed by replica set Created: 11/Oct/17  Updated: 08/Jan/24  Resolved: 02/Oct/19

Status: Closed
Project: Core Server
Component/s: Querying, Sharding
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: John Gibbons Assignee: Backlog - Query Team (Inactive)
Resolution: Duplicate Votes: 3
Labels: former-quick-wins, query-44-grooming
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-33597 Make allow_partial_results.js, return... Closed
Related
Assigned Teams:
Query
Operating System: ALL
Participants:
Case:

 Description   

The AllowPartialResults cursor method (DBQuery.Option.partial) does not have the desired effect when a shard backed by a REPLICA SET becomes unavailable.

In first example below I have set up a sharded cluster where each shard is backed by a single MONGOD. AllowPartialResults works as expected - if I kill on MONGOD I get results back from the other MONGOD. In the second example below, I have set up exact same cluster except I have backed each shard with a [single mongod] replica set. AllowPartialResults now does not work, in the sense that if I kill the mongod that forms the replica set, the db.collection.find().allowPartialResults() fails reporting an error saying "could not find host matching read preference { mode: \"primary\", tags: [ {} ] } for set shard1".



 Comments   
Comment by Esha Maharishi (Inactive) [ 02/Oct/19 ]

This was fixed (establishCursors was made to swallow FailedToSatisfyReadPreference errors from trying to find a particular host within a targeted shard) under SERVER-33597. Closing as duplicate.

Comment by Ramon Fernandez Marina [ 16/Nov/18 ]

Thanks for your report jg1 – this ticket has been sent to the Query team for evaluation. We'll post updates to the ticket as we find out more details about what's causing this behavior.

Regards,
Ramón.

Comment by John Gibbons [ 11/Oct/17 ]

I suspect the issue arises because the query does not get sent at all. Rather, the ReplicaSetMonitor (replica_set_monitor.cpp) fails in getHostOrRefresh (or possibly async_requests_sender.cpp) fails to connect to replica set before even the find gets sent.

Comment by John Gibbons [ 11/Oct/17 ]

THIS ONE PRETTY MUCH SAME EXCEPT SETS UP 2 SHARDS BACKED BY [single mongod] REPLICA SETS. partial DOES NOT HAVE DESIRED EFFECT IF ONE MONGOD KILLED.

kill -2 $(pgrep -a mongo | grep " 400" | awk '{print $1}')
sleep 1
cd
rm PartialTest2 -r -f
mkdir PartialTest2
cd PartialTest2/
mkdir config
mkdir shard1
mkdir shard2
mkdir logs
mongod --port 40001 --configsvr --replSet config --dbpath config --logpath logs/config.log --fork
mongo localhost:40001 --quiet --eval 'rs.initiate( { _id: "config", configsvr:true, members:[{_id:0, host: "localhost:40001"}]} )'
mongos --port 40002 --configdb config/localhost:40001 --logpath logs/mongos.log --fork
mongod --port 40011 --shardsvr --replSet shard1 --dbpath shard1 --logpath logs/shard1.log --fork
mongo localhost:40011 --quiet --eval 'rs.initiate( { _id: "shard1", members:[{_id:0, host: "localhost:40011"}]} )'
mongod --port 40012 --shardsvr --replSet shard2 --dbpath shard2 --logpath logs/shard2.log --fork
mongo localhost:40012 --quiet --eval 'rs.initiate( { _id: "shard2", members:[{_id:0, host: "localhost:40012"}]} )'
mongo localhost:40002 --quiet --eval 'sh.addShard("shard1/localhost:40011")'
mongo localhost:40002 --quiet --eval 'sh.addShard("shard2/localhost:40012")'
mongo localhost:40002 --quiet --eval 'sh.enableSharding("partial")'
mongo localhost:40002/partial --quiet --eval 'db.test.createIndex({"key" : 1})'
mongo localhost:40002 --quiet --eval 'sh.shardCollection("partial.test", { "key" : 1 })'
mongo localhost:40002 --quiet --eval 'sh.addShardTag("shard1", "firsthalf")'
mongo localhost:40002 --quiet --eval 'sh.addShardTag("shard2", "secondhalf")'
mongo localhost:40002 --quiet --eval 'sh.addTagRange("partial.test", { key : "A"}, {key: "L"}, "firsthalf")'
mongo localhost:40002 --quiet --eval 'sh.addTagRange("partial.test", { key : "L"}, {key: "Z"}, "secondhalf")'
mongo localhost:40002/partial --quiet --eval 'db.test.insert({ "key" : "B" })'
mongo localhost:40002/partial --quiet --eval 'db.test.insert({ "key" : "Y" })'
mongo localhost:40002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)' #should return 2
kill -2 $(pgrep -a mongo | grep "40011" | awk '{print $1}') db.test.find().addOption(DBQuery.Option.partial)
mongo localhost:40002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)' #should return 1 but fails

[mongod@BLAH PartialTest2]$ mongo localhost:40002/partial --quiet --eval 'db.test.find()'

{ "_id" : ObjectId("59dcc375f955bba2a5b15850"), "key" : "B" } { "_id" : ObjectId("59dcc375f90654100e5edb0e"), "key" : "Y" }

[mongod@BLAH PartialTest2]$ mongo localhost:40002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)'

{ "_id" : ObjectId("59dcc375f955bba2a5b15850"), "key" : "B" } { "_id" : ObjectId("59dcc375f90654100e5edb0e"), "key" : "Y" }

[mongod@BLAH PartialTest2]$ kill -2 $(pgrep -a mongo | grep "40011" | awk '

{print $1}

')

[mongod@BLAHPartialTest2]$ mongo localhost:40002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)'

Error: error: {

"ok" : 0,

"errmsg" : "could not find host matching read preference { mode: \"primary\", tags: [ {} ] } for set shard1",

"code" : 133,

"codeName" : "FailedToSatisfyReadPreference"

}

Comment by John Gibbons [ 11/Oct/17 ]

THIS ONE SETS UP 2 SHARDS BACKED DIRECTLY BY SINGLE MONGODS
AND partial WORKS FINE IF ONE MONGOD KILLED.

 
 
kill -2 $(pgrep -a mongo | grep " 300" | awk '{print $1}')
sleep 1
cd
rm PartialTest -r -f
mkdir PartialTest
cd PartialTest/
mkdir config
mkdir shard1
mkdir shard2
mkdir logs
mongod --port 30001 --configsvr --replSet config --dbpath config --logpath logs/config.log --fork
sleep 1
mongo localhost:30001 --quiet --eval 'rs.initiate( { _id: "config", configsvr:true, members:[{_id:0, host: "localhost:30001"}]} )'
sleep 5
mongos --port 30002 --configdb config/localhost:30001 --logpath logs/mongos.log --fork
mongod --port 30011 --shardsvr --dbpath shard1 --logpath logs/shard1.log --fork
mongod --port 30012 --shardsvr --dbpath shard2 --logpath logs/shard2.log --fork
mongo localhost:30002 --quiet --eval 'sh.addShard("localhost:30011")'
mongo localhost:30002 --quiet --eval 'sh.addShard("localhost:30012")'
mongo localhost:30002 --quiet --eval 'sh.enableSharding("partial")'
mongo localhost:30002/partial --quiet --eval 'db.test.createIndex({"key" : 1})'
mongo localhost:30002 --quiet --eval 'sh.shardCollection("partial.test", { "key" : 1 })'
mongo localhost:30002 --quiet --eval 'sh.addShardTag("shard0000", "firsthalf")'
mongo localhost:30002 --quiet --eval 'sh.addShardTag("shard0001", "secondhalf")'
mongo localhost:30002 --quiet --eval 'sh.addTagRange("partial.test", { key : "A"}, {key: "L"}, "firsthalf")'
mongo localhost:30002 --quiet --eval 'sh.addTagRange("partial.test", { key : "L"}, {key: "Z"}, "secondhalf")'
mongo localhost:30002/partial --quiet --eval 'db.test.insert({ "key" : "B" })'
mongo localhost:30002/partial --quiet --eval 'db.test.insert({ "key" : "Y" })'
mongo localhost:30002/partial --quiet --eval 'db.test.find()' #should return 2
mongo localhost:30002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)' #should return 2
kill -2 $(pgrep -a mongo | grep "30011" | awk '{print $1}')
mongo localhost:30002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)' #should return 1
mongo localhost:30002/partial --quiet --eval 'db.test.find()' #should error

Results:
mongos> db.test.find().addOption(DBQuery.Option.partial)

{ "_id" : ObjectId("59dcb0f38b49a86cdec9e780"), "key" : "B" } { "_id" : ObjectId("59dcb0f37950b2dc5ff66fd5"), "key" : "Y" }

mongos> exit

bye

[mongod@BLAH PartialTest]$ kill -2 $(pgrep -a mongo | grep "30011" | awk '

{print $1}

')

[mongod@BLAH PartialTest]$ mongo localhost:30002/partial --quiet --eval 'db.test.find().addOption(DBQuery.Option.partial)' #should return 1

{ "_id" : ObjectId("59dcb0f37950b2dc5ff66fd5"), "key" : "Y" }

[mongod@BLAH PartialTest]$ mongo localhost:30002/partial --quiet --eval 'db.test.find()' #should error because no partial

Error: error:

{ "ok" : 0, "errmsg" : "Connection refused", "code" : 6, "codeName" : "HostUnreachable" }
Generated at Thu Feb 08 04:27:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.