[COMPASS-5093] Opening a collection that has JSON schema validation fires off expensive query in the background Created: 15/Sep/21  Updated: 29/Oct/23  Resolved: 03/Feb/23

Status: Closed
Project: Compass
Component/s: UI / UX
Affects Version/s: None
Fix Version/s: 1.36.0

Type: Bug Priority: Critical - P2
Reporter: Juan Soto (Inactive) Assignee: Himanshu Singh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
Related
is related to COMPASS-4998 Opening a collection that has JSON sc... Closed
Story Points: 5
Documentation Changes: Needed
Documentation Changes Summary:

Schema Validation documentation mentions that the validation updates in real time:

> As you edit your validation, Compass updates in real-time to display a document from your collection that passes the validation and a document that fails.

This will not be happening anymore, you would need to press the button to get the preview documents that pass or fail validation

Sprint: Iteration Turtle, Iteration Unicornfish

 Description   

Problem Statement/Rationale

I have some very large collections (billions of documents) that have JSON schema validation in place.  If I open one of these collections in Compass, even if I don't click on the Validation tab it still fires off a very expensive query in the background to apparently find any documents that don't validate, e.g.

 

 

{ aggregate: \"mycollection\", pipeline: [ { $match: { $and: [ { $or: [
Unknown macro: { md5}

, { md5:

{ $exists: false }

} ] }, { $or: [

Unknown macro: { sha1}

, { sha1:

{ $exists: false }

} ] }, { $or: [

Unknown macro: { sha256}

, { sha256:

{ $exists: false }

} ] } ] } ] } }, { $group:

{ _id: 1, n: { $sum: 1 } }

} ], c..."

 

 

Steps to Reproduce

Create a collection with some data.  Add JSON schema validation.  Open the collection to the Documents tab in Compass and watch the query get fired off using db.currentOp().

Expected Results

This query should be triggered only if the users wants to execute. We should have some UI button to trigger this queries or something similar to the aggregate tab with sample docs. Or at least aggregation should use sample of 1000 docs. 

Actual Results

Even just opening the Documents tab of the collection in Compass causes this very expensive scan of the entire dataset.

Additional Notes

Any additional information that may be useful to include.



 Comments   
Comment by Githook User [ 01/Feb/23 ]

Author:

{'name': 'Himanshu Singh', 'email': 'himanshu.singhs@outlook.in', 'username': 'himanshusinghs'}

Message: Merge branch 'main' into COMPASS-5093-optimise-schema-validation-tab
Branch: COMPASS-5093-optimise-schema-validation-tab
https://github.com/mongodb-js/compass/commit/3d5366d7c1228b5ccddac9bd11fa7ae7d9f88aa8

Comment by Rhys Howell [ 16/Sep/21 ]

Hi juan.soto - thanks for bringing this up - we've fixed this behavior in https://jira.mongodb.org/browse/COMPASS-4998 which will land in the next Compass release (1.28.5). Now the validation queries are only executed when the validation tab is opened. Would you like to keep this ticket open as a request to make the validation queries only execute when a button is pressed on the validation tab, or can we close it?

Generated at Wed Feb 07 22:38:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.