[SERVER-21215] Shards should retry contacting config servers indefinitely Created: 30/Oct/15 Updated: 21/Jul/17 Resolved: 18/Nov/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 3.2.0-rc1 |
| Fix Version/s: | 3.2.0-rc4 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | John Page | Assignee: | Spencer Brody (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Minor Change | ||||||||
| Operating System: | ALL | ||||||||
| Sprint: | Sharding C (11/20/15) | ||||||||
| Participants: | |||||||||
| Description |
|
Root cause found - see comment 4 For testing I set up a 3.2-rc1 cluster with 32 shards. WT, Zlib - EVERYTHING on LOCALHOST I used a SINGLE , non replica set config server which rc1 allows me to do ( I raised a previous ticket that it should) I was able to successfully load 800GB of sharded data using a Java based loader. After restarting the cluster I get.
sh.status appears to work OK Show collections gives
All mongod logs show
I can connect to the config server with mongo on localhost:27019 |
| Comments |
| Comment by Andy Schwerin [ 20/Nov/15 ] | |||||||||||||||||||||||||||||||||||
|
Documentation should note that in 3.2, mongos during startup will continue trying to contact config servers until one becomes available, instead of shutting down if it cannot find a config server to talk to. | |||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 18/Nov/15 ] | |||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'stbrody', u'name': u'Spencer T Brody', u'email': u'spencer@mongodb.com'}Message: | |||||||||||||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 30/Oct/15 ] | |||||||||||||||||||||||||||||||||||
|
Discussed live with Andy and Kal, who explained there are three bugs involved here. Since | |||||||||||||||||||||||||||||||||||
| Comment by John Page [ 30/Oct/15 ] | |||||||||||||||||||||||||||||||||||
|
Also mongod doesn't quit - just sits there running but refusing connections which would then not show up in any startup scripts. | |||||||||||||||||||||||||||||||||||
| Comment by John Page [ 30/Oct/15 ] | |||||||||||||||||||||||||||||||||||
|
SOLVED - ISH It appears that in 3.2 if you have a mongod which is a shard - then you cannot start it unless the config servers are up and running. Previous to 3.2 you could startup the mongod servers first, then the config server then the mongos. You can still do this when first creating the cluster. This is a significant behaviour change which could cause custoemr clusters that are not always on to fail to start - or a cluster to fail to start after a full reboot. Therefore I'm still going to claim it's a bug. | |||||||||||||||||||||||||||||||||||
| Comment by John Page [ 30/Oct/15 ] | |||||||||||||||||||||||||||||||||||
|
Full log
| |||||||||||||||||||||||||||||||||||
| Comment by John Page [ 30/Oct/15 ] | |||||||||||||||||||||||||||||||||||
|
MongoD is running on 27102
mongo shell CANNOT connect to it though |