[SERVER-5937] Remove document size limits by transparently splitting it (at least for arrays) Created: 26/May/12  Updated: 06/Dec/22

Status: Open
Project: Core Server
Component/s: Performance, Storage
Affects Version/s: 2.1.1
Fix Version/s: features we're not sure of

Type: Improvement Priority: Minor - P4
Reporter: John Wood Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 4
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

All Platforms


Issue Links:
Related
related to SERVER-12305 Allow command request and response BS... Backlog
is related to SERVER-432 server side expansion of DBRefs Closed
Assigned Teams:
Query Execution
Participants:

 Description   

To work around the current document size limit, if there is any chance a document or its children may exceed this limit then the nested children must be split out to separate collections and somehow linked.

MongoDB could do this automatically and transparently. If an array has a specific flag/attribute, Mongo will store the array in a separate collection, with a prefix to avoid namespace conflicts. Each document in the new collection will be given an internal parent or path field to allow it to be linked to the original, parent document. The original document will contain a proxy array field to replace the array. The proxy will contain a link to the collection where the actual array is stored.

MongoDB will then have to rewrite any queries, splitting them up into separate queries for each collection and providing a cursor that can traverse the merged results.

MongoDB would also have to rewrite and split out inserts, updates and deletes to properly allocate changes to the appropriate collection.

Because of its schemaless design, it may be easier to define this similarly to the way indexes are defined, rather than just a flag in the actual array. That way it won't have to wait until it finds the proxy in the physical document before determining which indexes it should be using.


Generated at Thu Feb 08 03:10:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.