[SERVER-21906] Race in ShardRegistry::reload and config.shard update can cause shard not found error Created: 15/Dec/15 Updated: 21/Nov/16 Resolved: 06/Jan/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 3.2.0 |
| Fix Version/s: | 3.2.3, 3.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Randolph Tan | Assignee: | Misha Tyulenev |
| Resolution: | Done | Votes: | 0 |
| Labels: | code-only | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Completed: | |||||||||||||||||
| Sprint: | Sharding E (01/08/16) | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 0 | ||||||||||||||||
| Description |
|
The ReplicaSetMonitor monitors membership changes of all the known replica set shards in mongos and tries to update the ShardRegistry and config.shards. The issue is that when the ShardRegistry::reload gets called, it will clear all it's entries and tries to reread the contents from config.shards. So there is a small window were the ShardRegistry 'forgets' the newly added replSet member until config.shards get updated and reload gets called again. |
| Comments |
| Comment by Githook User [ 19/Jan/16 ] |
|
Author: {u'username': u'mikety', u'name': u'Misha Tyulenev', u'email': u'misha@mongodb.com'}Message: (cherry picked from commit a05ba516fb340bd7a22480303c8d647b6d6c9874) |
| Comment by Githook User [ 06/Jan/16 ] |
|
Author: {u'username': u'mikety', u'name': u'Misha Tyulenev', u'email': u'misha@mongodb.com'}Message: |
| Comment by Spencer Brody (Inactive) [ 15/Dec/15 ] |
|
Proposed solution is to change how ShardRegistry::reload() works so that it always takes the list of hosts from the ReplicaSetMonitor (via the Targeter) rather than from the contents of config.shards. config.shards is only used to seed new Targeters for newly-discovered shards. |