[SERVER-36427] Sharding cluster unable to sync and mongodump fails. Created: 03/Aug/18 Updated: 15/Sep/18 Resolved: 21/Aug/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication, Sharding |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Prasad Surase | Assignee: | Nick Brewer |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
I have a 13 server mongodb cluster consisting of 1 query router, 3 config servers with replication and 3 shards, each with replication(primary, secondary and arbiter). Its installed on AWS-EC2 R series instances. Monit is used to restart the mongodb service incase it exceeds 95% memory usage. My Shard3Primary failed and the Shard3Secondary became primary(as expected). The problem is that the Shard3Primary mongodb process isnt able to restart stating
I tried to take the mongodump on the QueryRouter and it failed too.(the same command had succeeded for earlier dumps). I have attached the screenshots of Shard3Primary for your reference. |
| Comments |
| Comment by Nick Brewer [ 21/Aug/18 ] |
|
The fix for this behavior has been released in MongoDB 3.6.7. -Nick |
| Comment by Nick Brewer [ 03/Aug/18 ] |
|
prasadsurase I believe you're running into a known issue that is detailed here: A fix to prevent this behavior is in MongoDB 4.0, but it hasn't been backported to 3.6 yet. In the meantime, you'll need to perform an initial sync to restore the primary. I'll update this ticket once the fix is introduced in 3.6. -Nick |