Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31747

Allow projections inside lookup stages in aggregation pipelines

    • Type: Icon: New Feature New Feature
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Aggregation Framework
    • Labels:
    • Query
    • Major Change

      Now when you have a lookup stage in your aggregation pipeline, that lookup retrieves all the complete documents from the foreign collection, this is risky because you can reach the maximum document size very easily if:

      1. The foreign collection is big enough
      2. Each document is very heavy (our case)

      It may be legit to increase the maximum document size limit when you need all the data from the foreign collection documents, but you will finally reach the limit at some point, and, for the second point, not always all the data holding from a document is needed, just a small portion of it, but, if you cannot make a projection from each document, discarding what you don't need, then the maximum document size will be reached as well at some point. In fact the limit could be always reached, but it is harder to reach that limit when you have a bunch of numbers, than having a bunch of other huge data.

      We think that the solution is to allow projections inside the lookup stage.

            Assignee:
            backlog-server-query Backlog - Query Team (Inactive)
            Reporter:
            jgl Jose
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: