[SERVER-28268] Allow for collections with only immutable docs to reduce memory usage for snapshots in WT. Created: 10/Mar/17  Updated: 27/Oct/23  Resolved: 10/Mar/17

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: James Reitz Assignee: Backlog - Storage Execution Team
Resolution: Works as Designed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-8315 Immutable Collections Closed
Assigned Teams:
Storage Execution
Participants:

 Description   

We have a number of collections whose documents are never modified. It seems like Mongodb should support a feature where a collection could be marked as "doc-immutable", meaning that documents are not updatable (but would be deletable). Then when performing queries, a snapshot would not be needed by WT to return query results. I would imagine that this could save a great deal of memory in the WT cache, especially for queries that bring back many documents.



 Comments   
Comment by Eric Milkie [ 10/Mar/17 ]

No problem! We welcome all improvement design ideas.

Comment by James Reitz [ 10/Mar/17 ]

Thanks Eric. After I submitted the feature request, I read up on MVCC, and after learning about it, I thought this feature may not be worthwhile. Thanks for the quick response though.

Comment by Eric Milkie [ 10/Mar/17 ]

WiredTiger does not make a full copy of all data to implement snapshots, so snapshot use does not appreciably increase memory usage. A copy of a record is kept for each record update, so if a record is never updated, there should be only one copy in the cache.
For a given query, all results must be returned from pages in the cache, so it wouldn't be possible to skip populating the cache when querying documents without a larger design change.

Comment by Rob Manning [ 10/Mar/17 ]

What James Reitz (Jim is a member of my team) may have omitted is that we still need the ability to insert and delete documents in these collections (but 0 updates). I'm not sure about inserts, but deletions may be a problem if performed by some clients while others are attempting to query a consistent view of the collection. Perhaps it would be easier to implement this feature request by adding a restriction so that deletions may only be performed by adding a TTL index to any immutable collection (perhaps in an immutable database if that is also necessary). With that restriction, MongoDB has full control of when documents are deleted from cache to provide consistency to querying clients. The insert case might be a different challenge.

Comment by Kelsey Schubert [ 10/Mar/17 ]

Hi jimreitz,

Thanks for the feature request. I've marked it to be considered by the storage team, please continue to watch for updates.

Kind regards,
Thomas

Comment by James Reitz [ 10/Mar/17 ]

By the way, this is a feature request.

Generated at Thu Feb 08 04:17:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.