Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5937

Remove document size limits by transparently splitting it (at least for arrays)

    • Query Execution

      To work around the current document size limit, if there is any chance a document or its children may exceed this limit then the nested children must be split out to separate collections and somehow linked.

      MongoDB could do this automatically and transparently. If an array has a specific flag/attribute, Mongo will store the array in a separate collection, with a prefix to avoid namespace conflicts. Each document in the new collection will be given an internal parent or path field to allow it to be linked to the original, parent document. The original document will contain a proxy array field to replace the array. The proxy will contain a link to the collection where the actual array is stored.

      MongoDB will then have to rewrite any queries, splitting them up into separate queries for each collection and providing a cursor that can traverse the merged results.

      MongoDB would also have to rewrite and split out inserts, updates and deletes to properly allocate changes to the appropriate collection.

      Because of its schemaless design, it may be easier to define this similarly to the way indexes are defined, rather than just a flag in the actual array. That way it won't have to wait until it finds the proxy in the physical document before determining which indexes it should be using.

            Assignee:
            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            Reporter:
            maverin John Wood
            Votes:
            4 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated: