[SERVER-80346] validate doesn't validate BSONColumn's binary format Created: 23/Aug/23  Updated: 29/Oct/23  Resolved: 20/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Matt Kneiser Assignee: Damian Wasilewicz
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-81206 BSONColumn is decompressed several ti... Open
Assigned Teams:
Storage Execution NAMER
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Execution NAMR Team 2023-09-18, Execution NAMR Team 2023-10-02
Participants:

 Description   

We do basic validation on time-series collections but we don't check the binary format itself to see if it's corrupted. Validate merely checks for BinDataType 7.



 Comments   
Comment by Githook User [ 20/Sep/23 ]

Author:

{'name': 'Damian Wasilewicz', 'email': '33820523+DamianWasilewicz@users.noreply.github.com', 'username': 'DamianWasilewicz'}

Message: SERVER-80346 Move BSONColumn decompression behind checkBSONConformance option for the validate command
Branch: master
https://github.com/mongodb/mongo/commit/4faea7104090dcbbd5196dade68cf23e1e591240

Comment by Yuhong Zhang [ 07/Sep/23 ]

Giving it another look, we actually decompress the BSONColumns in time-series buckets documents by default when running a validate command (here, here). We might want to use this ticket to put it behind the full:true option, adding more comments, and handle the exception better when it fails.

Comment by Yuhong Zhang [ 28/Aug/23 ]

We perform BSON structural validation before validating the time-series specific properties. For the BSONColumn format, there isn't much we can do other than trying to decompress it and see if an exception is thrown. This is also relatively expensive compared to other structural checks so we put it behind checkBSONConformance: true or full:true for the validate command.

I think now that BSON bucket documents are always compressed, we might want to consider including this check by default when running the validate command.

cc damian.wasilewicz@mongodb.com 

Comment by Matt Kneiser [ 23/Aug/23 ]

Upon further inspection it looks like there's actual validation of most items in the data field:
timestamps & min/max

I'm going to keep this ticket open for any gaps or improvements desired.

Generated at Thu Feb 08 06:43:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.