[SERVER-80346] validate doesn't validate BSONColumn's binary format Created: 23/Aug/23 Updated: 29/Oct/23 Resolved: 20/Sep/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Matt Kneiser | Assignee: | Damian Wasilewicz |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Storage Execution NAMER
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Sprint: | Execution NAMR Team 2023-09-18, Execution NAMR Team 2023-10-02 | ||||||||
| Participants: | |||||||||
| Description |
|
We do basic validation on time-series collections but we don't check the binary format itself to see if it's corrupted. Validate merely checks for BinDataType 7. |
| Comments |
| Comment by Githook User [ 20/Sep/23 ] |
|
Author: {'name': 'Damian Wasilewicz', 'email': '33820523+DamianWasilewicz@users.noreply.github.com', 'username': 'DamianWasilewicz'}Message: |
| Comment by Yuhong Zhang [ 07/Sep/23 ] |
|
Giving it another look, we actually decompress the BSONColumns in time-series buckets documents by default when running a validate command (here, here). We might want to use this ticket to put it behind the full:true option, adding more comments, and handle the exception better when it fails. |
| Comment by Yuhong Zhang [ 28/Aug/23 ] |
|
We perform BSON structural validation before validating the time-series specific properties. For the BSONColumn format, there isn't much we can do other than trying to decompress it and see if an exception is thrown. This is also relatively expensive compared to other structural checks so we put it behind checkBSONConformance: true or full:true for the validate command. I think now that BSON bucket documents are always compressed, we might want to consider including this check by default when running the validate command. |
| Comment by Matt Kneiser [ 23/Aug/23 ] |
|
Upon further inspection it looks like there's actual validation of most items in the data field: I'm going to keep this ticket open for any gaps or improvements desired. |