[COMPASS-422] mongodb-schema: provide a memory limit parameter Created: 22/Nov/16  Updated: 10/Jan/24  Resolved: 27/Sep/19

Status: Closed
Project: Compass
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Thomas Rueckstiess Assignee: Unassigned
Resolution: Incomplete Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to COMPASS-367 Configurable sample size Closed
Epic Link: COMPASS-2234

 Description   

mongodb-schema should take a memory limit parameter and stop analysing the schema when that limit is reached.

It could either (a) stop adding values and simply update counts/probabilities, or (b) stop sampling documents alltogether.

It should probably also contain a top-level field "memoryLimitReached: true", so that Compass can give the user some feedback.

This work has already been started as part of COMPASS-218 but is too risky to merge for 1.5 stable. Re-visit after 1.5 was released.



 Comments   
Comment by Massimiliano Marcon [ 27/Sep/19 ]

Closing this issue in JIRA. It's been moved to the new, unified feedback engine at https://feedback.mongodb.com/forums/924283-compass.

We are looking for input to decide how to prioritize it, so go there, look for the suggestion, vote and leave comments. If you don't find it, feel free to create a new one.

Comment by Brian Blevins [ 29/Mar/17 ]

Customer feedback on Compass memory utilization.

00432392: Using too much memory for large collection
https://mongodb.my.salesforce.com/500A000000Wkxd0
https://support.mongodb.com/case/00432392

2017-July-17:
The customer in that case came back with the following info.

We are now using version 1.7.1 and noticed that it is faster than before (sorry, I do not recall which version we had back in March, 2017). It used to take more than 30 minutes and longer. It takes about 7 minutes now.

We are running it on Windows 7 workstation pointing to MongoDB replica set running under Red Hat Linux. We are running version 3.2.8-ent MongoDB.

Comment by Thomas Rueckstiess [ 22/Nov/16 ]

Current (unfinished) implementation: https://github.com/mongodb-js/mongodb-schema/tree/COMPASS-422-memory-limit

Comment by Githook User [ 22/Nov/16 ]

Author:

{'name': 'Thomas Rueckstiess', 'email': 'thomas@rueckstiess.net', 'username': 'rueckstiess'}

Message: COMPASS-422 [wip] add memory limit. needs tests.
Branch: COMPASS-422-memory-limit
https://github.com/mongodb-js/mongodb-schema/commit/610bfc8e80908afe4097d36649275ace51e7b0db

Generated at Wed Feb 07 22:25:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.