Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5937

Remove document size limits by transparently splitting it (at least for arrays)

    XMLWordPrintable

    Details

      Description

      To work around the current document size limit, if there is any chance a document or its children may exceed this limit then the nested children must be split out to separate collections and somehow linked.

      MongoDB could do this automatically and transparently. If an array has a specific flag/attribute, Mongo will store the array in a separate collection, with a prefix to avoid namespace conflicts. Each document in the new collection will be given an internal parent or path field to allow it to be linked to the original, parent document. The original document will contain a proxy array field to replace the array. The proxy will contain a link to the collection where the actual array is stored.

      MongoDB will then have to rewrite any queries, splitting them up into separate queries for each collection and providing a cursor that can traverse the merged results.

      MongoDB would also have to rewrite and split out inserts, updates and deletes to properly allocate changes to the appropriate collection.

      Because of its schemaless design, it may be easier to define this similarly to the way indexes are defined, rather than just a flag in the actual array. That way it won't have to wait until it finds the proxy in the physical document before determining which indexes it should be using.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-query Backlog - Query Team
              Reporter:
              maverin John Wood
              Participants:
              Votes:
              4 Vote for this issue
              Watchers:
              10 Start watching this issue

                Dates

                Created:
                Updated: