[SERVER-48081] Malformed document validators can cause a secondary to fail on oplog application Created: 11/May/20 Updated: 22/Jan/21 Resolved: 22/Jan/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Nicholas Zolnierz | Assignee: | Ted Tuckman |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | qopt-team | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Query 2020-11-30, Query 2020-12-14, Query 2020-12-28, Query 2021-01-11, Query 2021-01-25 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Participants: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Found as part of Note that there are two distinct paths to a failure. One is the case described above, where a node is catching up after upgrade and encounters the malformed doc validator. The second is when the node starts up and already has the entry for the "create". In that case, it will happily build the in-memory Collection state with a non-OK validator, however any subsequent inserts to that collection will fail on oplog application. |
| Comments |
| Comment by Charlie Swanson [ 15/Oct/20 ] |
|
ted.tuckman if you have some time next sprint can you pick up the investigation that katherine.wu started? Lower priority than all the other things in your sprint. |
| Comment by Nicholas Zolnierz [ 11/May/20 ] |
|
max.hirschhorn if the Collection state already has a non-OK validator, then we fail before checking DisableDocumentValidation. It does seem like we should move that check up to the top of the function which AFAICT would fix the insert issue. |
| Comment by Max Hirschhorn [ 11/May/20 ] |
Secondaries doing oplog application have always used DisableDocumentValidation to skip doing document validation. Are you suggesting there were cases where that wasn't sufficient? |
| Comment by Pierlauro Sciarelli [ 11/May/20 ] |
|
Take a look to this commit relative to SERVER-47748. Validation can be disabled by using UnreplicatedWritesBlock. |
| Comment by Nicholas Zolnierz [ 11/May/20 ] |
|
kaloian.manassiev oh that's interesting, do you have a ticket #? |
| Comment by Kaloian Manassiev [ 11/May/20 ] |
|
FYI pierlauro.sciarelli made it so that validators will not run on secondaries at all. Prior to that change, we used to run both validation and shard version checking. |