[SERVER-53176] Return an error when commitQuorum includes voting buildIndexes:false members Created: 02/Dec/20 Updated: 29/Oct/23 Resolved: 13/Jan/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance, Replication |
| Affects Version/s: | 4.4.2 |
| Fix Version/s: | 4.9.0, 4.4.4 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Konrad Baumgart | Assignee: | Louis Williams |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
mongo db server and cli client 4.4.2 |
||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Backport Requested: |
v4.4
|
||||||||||||||||||||
| Sprint: | Execution Team 2021-01-25 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
Given a replica set with config like
Building indexes on replica-set with default commitQuorum (votingMembers) seems to wait for build on member that doesn't build indexes (due to replica-set configuration) I have to lower commitQuorum to majority, otherwise createIndex command doesn't finish. votingMembers commitQuorum should mean: all data-bearing index-building voting replica set members. |
| Comments |
| Comment by Githook User [ 15/Jan/21 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Louis Williams', 'email': 'louis.williams@mongodb.com', 'username': 'louiswilliams'}Message: (cherry picked from commit eed69840da689b56db5f4f7fc0da991c27f28fb6) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 13/Jan/21 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Louis Williams', 'email': 'louis.williams@mongodb.com', 'username': 'louiswilliams'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Louis Williams [ 11/Jan/21 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Great! When determining whether a commit quorum is satisfiable, we will not consider voting, buildIndexes:false nodes.
True, making your hidden node non-voting means that your replica set will not have a majority if one of the other nodes fails. The other solution would be to add another regular node to your replica set so that you have 3 eligible primaries and 1 hidden node, cost permitting. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Konrad Baumgart [ 11/Jan/21 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
I have only 3 nodes, so marking one of them as non-voting would make cluster less resilient. I handled my problem by modifying the lib like:
Throwing an error seems to be good solution, as it would help both future users and me (when executing commands via cli and forgetting about that commit qourum). It would be best to handle other commit quorums too: in my config using commitQurum 3 is also problematic and in some ill configurations "majority" may be also problematic. I see there is already some check implemented for simplest unsatisfiable commit quorum, so extending it seems good.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Louis Williams [ 08/Jan/21 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi, konrad.baumgart@erli.pl. I understand that this is confusing and undesirable behavior. To avoid the workaround in your code, you could reconfigure the buildIndexes: false node to be non-voting. Documentation here. That node is already hidden and has a priority of 0. If you set votes to also be 0, then it will not be included in the default commitQuorum. This means your createIndexes commands can remain topology-agnostic for testing. Rather than changing the meaning of "votingMembers" or even editing the documentation, I think the most reasonable solution would be to prevent future users from running into this problem. That is, we should return an error when a user tries to create an index with the "votingMembers" commit quorum and there are voting, buildIndexes: false nodes in the set. Otherwise, it is guaranteed that your index build will never complete. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Konrad Baumgart [ 21/Dec/20 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
For me (and hopefully most other use-cases) it would be best, if I could call
no matter if I'm on replicaset or standalone. Currently I have to write code like
to run code both locally and in integration tests and in production without error 'Standalones can't specify commitQuorum' | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Eric Sedor [ 02/Dec/20 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks konrad.baumgart@erli.pl, I am able to reproduce this:
It may not be that modifying the meaning of votingMembers is the best solution to this problem, but it is clear that marking a voting member with buildIndexes: false causes the default commitQuorum. I'll pass this on to an appropriate team to consider this behavior. As you've already noted, if buildIndexes is set, you must also manually relax the commitQuorum to a number or setting that allows the index-building nodes to satisfy quorum. Gratefully, |