[SERVER-25205] MongoDB overwrites version field in replSetReconfig if "force" is given Created: 21/Jul/16 Updated: 13/Aug/16 Resolved: 02/Aug/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 3.2.7 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Joseph Glanville | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Steps To Reproduce: | Run replSetReconfig with an explicit version in the config document |
| Participants: |
| Description |
|
When trying to explicitly set the version of the replica set configuration the server will overwrite the version specified by the operator if the "force" flag is given. Instead the version number will be incremented randomly by a relatively large number. This behaviour is undesirable if one intends to build a system to automatically update the replica set configuration as in these cases it's important the external system is able to set the version to a known number. It is the job of that external system to ensure it's monotonically increasing etc. Ideally the behaviour would be changed to not overwrite operator input and instead like the non-force path should return an error if the version number is less than the old version. If I have missed something vital that explains why the server does this I would love to hear the rational behind this behaviour. |
| Comments |
| Comment by Joseph Glanville [ 13/Aug/16 ] |
|
I see this has been closed as Works As Designed but I think the problem here is that this behavior eliminates the possibility of a fully safe automated orchestration system for MongoDB without compiling a custom build without this random increment. I posted this to the list but didn't get any additional replies but it summarises my point fairly concisely: I agree the random increment is probably a useful measure in the case that an administrator is manually triggering a force reconfigure and there is the chance for operator error to issue this against more than one peer. However my use-case is very different. I have an external system that has complete knowledge of all MongoDB peers, oplog positions, etc and could ensure that only monotonically increasing replSetConfig version numbers are used. Being able to apply all reconfigures with known version numbers greatly increases the safety of my usecase because otherwise if my system does actually issue a reconfigure using stale information due to a race condition that it will be properly rejected by the cluster. Without this functionality such a system can't be made fully safe. Would it be possible to implement a means to circumvent this random increment in the case that you really do understand the consequences? |
| Comment by Kelsey Schubert [ 02/Aug/16 ] |
|
Hi josephglanville, I see you also posted this question on the mongodb-dev user group where Dwight provided the rational for this behavior:
To summarize, this is expected behavior and provides some protection against two replica set members receiving forced configs at the same time. We recommend that {force: true} be used only in emergencies and advise against using it as part of a script. For MongoDB-related support discussion please post on the mongodb-users group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-users group. Kind regards, |
| Comment by Joseph Glanville [ 21/Jul/16 ] |
|
I couldn't find an edit button but to elaborate on why I need to be able to use the "force" flag is that occasionally the system needs to recover the replica set from states where a quorum of members is no longer available and will often need to be used on non-primary peers. |