[SERVER-46720] Make replication internal message non-strict parsing Created: 09/Mar/20 Updated: 29/Oct/23 Resolved: 16/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.0-rc4, 4.7.0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Siyuan Zhou | Assignee: | William Schultz (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | safe-reconfig-related | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Backport Requested: |
v4.4, v4.2
|
||||||||||||
| Sprint: | Repl 2020-04-06, Repl 2020-04-20 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
Since we have IDL for most of the messages, internal messages don't need strict parsing. It causes extra work on upgrade / downgrade procedures. |
| Comments |
| Comment by Githook User [ 01/May/20 ] |
|
Author: {'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}Message: (cherry picked from commit 0793797e87ea2c9c6e0a92f5fd704da15ffc0e51) |
| Comment by Githook User [ 01/May/20 ] |
|
Author: {'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}Message: (cherry picked from commit 9c5d82ff1fb2bb7367e10c9e21ac67b440e7097e) |
| Comment by Githook User [ 16/Apr/20 ] |
|
Author: {'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}Message: |
| Comment by Githook User [ 10/Apr/20 ] |
|
Author: {'name': 'William Schultz', 'email': 'william.schultz@mongodb.com', 'username': 'will62794'}Message: |
| Comment by Judah Schvimer [ 27/Mar/20 ] |
This is a good point, thank you! I think you're right that ISS doesn't have a problem with non-strict on-disk parsing. I don't see how SERVER-47007 benefits from non-strict parsing, though. My understanding of this ticket is that nodes will ignore extra fields, not that they will ignore missing fields. SERVER-47007 requires removing a field, not adding one. Either way, I'm fine relaxing on-disk parsing since it won't complicate ISS. |
| Comment by Siyuan Zhou [ 26/Mar/20 ] |
I thought we will do a reconfig on FCV downgrade, so I don't see why downgrade relies on the fassert.
If they don't follow the downgrade procedures, would they have FCV 4.6? If so, starting binary 4.4 with FCV 4.6 will fail loudly. Without non-strict parsing, any change to the on-disk format will be a backward compatible problem. LastVote in SERVER-47007 is an example as mentioned above. We'd have to explicitly update LastVote on FCV downgrade otherwise. |
| Comment by Judah Schvimer [ 26/Mar/20 ] |
|
Initial Sync Semantics downgrade design relies on 4.4 nodes fasserting when they see a "newlyAdded" field, since they won't know how to interpret it correctly. If users follow proper downgrade procedures it shouldn't be a problem, but we were relying on this for when they don't. I had previously thought that we would only relax network message parsing, not on disk parsing. I think in this case it's fine but I will have to think more to determine if it's safe if we choose to relax on disk parsing, or if it will complicate ISS downgrade. I'm nervous about nodes ignoring data on disk. |
| Comment by Siyuan Zhou [ 26/Mar/20 ] |
|
judah.schvimer, I think so. Do you have any concern? |
| Comment by Judah Schvimer [ 26/Mar/20 ] |
|
Did we decide on if this would include relaxing on disk parsing? |
| Comment by Siyuan Zhou [ 12/Mar/20 ] |
|
LastVote is another example, strict parsing makes it hard to record the candidate ID rather than the candidate index. |
| Comment by Siyuan Zhou [ 12/Mar/20 ] |
|
We should also consider relax the constraints on on-disk format to make future upgrade/downgrade work easier or introduce new fields in minor versions. For example, that will let us replace replica set name with replica set ID to support replset renaming or allow alias for slaveDelay without upgrade / downgrade concerns. |