[COMPASS-4998] Opening a collection that has JSON schema validation fires off expensive query in the background Created: 10/Aug/21  Updated: 29/Oct/23  Resolved: 01/Sep/21

Status: Closed
Project: Compass
Component/s: Compass
Affects Version/s: 1.28.1
Fix Version/s: 1.29.4

Type: Bug Priority: Critical - P2
Reporter: Michael Wilson Assignee: Le Roux Bodenstein
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to COMPASS-5093 Opening a collection that has JSON sc... Closed
Story Points: 5
Documentation Changes: Not Needed
Sprint: Iteration Johannesburg, Iteration Kampala

 Description   

Problem Statement/Rationale

I have some very large collections (billions of documents) that have JSON schema validation in place.  If I open one of these collections in Compass, even if I don't click on the Validation tab it still fires off a very expensive query in the background to apparently find any documents that don't validate, e.g.

 

{ aggregate: \"mycollection\", pipeline: [ { $match: { $and: [ { $or: [ { md5: { $type: \"string\", $regex: \"^[a-f0-9]{32}$\" } }, { md5: { $exists: false } } ] }, { $or: [ { sha1: { $type: \"string\", $regex: \"^[a-f0-9]{40}$\" } }, { sha1: { $exists: false } } ] }, { $or: [ { sha256: { $type: \"string\", $regex: \"^[a-f0-9]{64}$\" } }, { sha256: { $exists: false } } ] } ] } ] } }, { $group: { _id: 1, n: { $sum: 1 } } } ], c..."

 

Steps to Reproduce

Create a collection with some data.  Add JSON schema validation.  Open the collection to the Documents tab in Compass and watch the query get fired off using db.currentOp().

Expected Results

Really this should never happen as with any collections of significant size this is going to cause an issue.  I'm not sure what it's trying to populate in the UI, but it's probably better to allow the user to initiate an action to pull this result.  At the very least, it shouldn't happen until I open the Validation tab, so that I can avoid it by just not going into that part of the UI.

Actual Results

Even just opening the Documents tab of the collection in Compass causes this very expensive scan of the entire dataset.

Additional Notes

Any additional information that may be useful to include.



 Comments   
Comment by Githook User [ 02/Sep/21 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix: lazy-load validations when the user browses to the validation tab (COMPASS-4998) (#2433)

  • fix: lazy-load validations when the user browses to the validation tab
  • fix is-loaded and add tests
  • tweak dispatch order
Comment by Githook User [ 01/Sep/21 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix: lazy-load validations when the user browses to the validation tab (COMPASS-4998) (#2433)

  • fix: lazy-load validations when the user browses to the validation tab
  • fix is-loaded and add tests
  • tweak dispatch order
Generated at Wed Feb 07 22:38:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.