[SERVER-43873] MR Agg: assert if the size of an emitted value exceeds 1/2 of the max BSON size Created: 07/Oct/19  Updated: 27/Oct/23  Resolved: 29/Oct/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Nicholas Zolnierz Assignee: James Wahlin
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to DOCS-13179 Remove MapReduce emit restriction for... Closed
Sprint: Query 2019-11-04
Participants:

 Comments   
Comment by James Wahlin [ 29/Oct/19 ]

Closing this ticket as we are not going to add this restriction for the aggregation based mapReduce command. DOCS-13179 has been filed to update our documentation accordingly.

Comment by James Wahlin [ 29/Oct/19 ]

I spent some time testing this scenario and have come to the opinion that we should not add back this limitation directly. I don't see a concrete reason to enforce this for an aggregation-based mapReduce and could imagine successful mapReduce invocations where an emitted value exceeds 1/2 max bson object size.

Additionally we have a different check that enforces that the set of emitted values for an input document doesn't exceed 100MB.

nicholas.zolnierz - given the above I am leaning towards making a documentation change. I would be happy to debate further though.

Comment by David Storch [ 08/Oct/19 ]

Got it, thanks for the context.

Comment by Nicholas Zolnierz [ 08/Oct/19 ]

david.storch, I spoke with James briefly about this and realize now I should've included a description. The current MR enforces this oddly enough, but (more importantly) it's also documented it as a limitation. Digging through the git blame goes back about 10 years so presumably there was a valid reason back then.

We're going to discuss it at the MR standup and decide whether it's worth preserving or just file a DOCS change.

Comment by David Storch [ 08/Oct/19 ]

nicholas.zolnierz is this consistent with the behavior of the old mapReduce implementation? I'm not sure what's motivating the behavior described by the title.

Generated at Thu Feb 08 05:04:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.