[SERVER-31747] Allow projections inside lookup stages in aggregation pipelines Created: 27/Oct/17 Updated: 06/Dec/22 Resolved: 27/Oct/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Jose | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | feature | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query
|
||||||||
| Backwards Compatibility: | Major Change | ||||||||
| Participants: | |||||||||
| Description |
|
Now when you have a lookup stage in your aggregation pipeline, that lookup retrieves all the complete documents from the foreign collection, this is risky because you can reach the maximum document size very easily if:
It may be legit to increase the maximum document size limit when you need all the data from the foreign collection documents, but you will finally reach the limit at some point, and, for the second point, not always all the data holding from a document is needed, just a small portion of it, but, if you cannot make a projection from each document, discarding what you don't need, then the maximum document size will be reached as well at some point. In fact the limit could be always reached, but it is harder to reach that limit when you have a bunch of numbers, than having a bunch of other huge data. We think that the solution is to allow projections inside the lookup stage. |
| Comments |
| Comment by James Wahlin [ 27/Oct/17 ] |
|
Hi Jose, The feature you request will be available in MongoDB 3.6.0 and can be tested in our 3.6.0-rc1 release candidate. This comes as part of a new $lookup feature which allows users to specify a foreign pipeline. For details see the following docs page: Thanks, |
| Comment by Mark Agarunov [ 27/Oct/17 ] |
|
Hello jgl, Thank you for the detailed description. I've set the fixVersion to "Needs Triage" for this new feature to be scheduled against our currently planned work. Updates will be posted on this ticket as they happen. Thanks, |