[SERVER-46923] $unionWith doesn't support null pipelines Created: 17/Mar/20  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Varsha Subrahmanyam (Inactive) Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Optimization
Participants:

 Description   

Mongoast considers the following syntaxes to be equivalent:

{ $unionWith: "foo" }
{ $unionWith: { coll: "foo" }}
{ $unionWith: { coll: "foo", pipeline: null } }

The third case is currently unsupported by the server, though, because null pipelines are not allowed.



 Comments   
Comment by James Wahlin [ 17/Apr/20 ]

I spent some time looking at how we handle null arguments in other commands and aggregation stages and found that we do not equate null with missing. Examples of this are:
1) A "collation: null" argument to the find command results in: "Failed to parse: collation: null. 'collation' field must be of BSON type object."
2) A "min: null" argument to the find command results in: "Failed to parse: min: null. 'min' field must be of BSON type object."
3) A "let: null" argument to a $lookup stage results in: "$lookup argument 'let: null' must be an object, is type null"
4) A "output: null" argument to a $bucketAuto stage results in: "The $bucketAuto 'output' field must be an object, but found type: null"

Given that in addition to our current plan to not treat null and missing in the same manner, I am inclined to say we should make no changes at this point in time. Rather than close this ticket however, how about we backlog and make part of PM-1648, "Distinguish Null and Missing in Indexes"? We can give deeper thought to this under that project and would be in a position to make broader changes if we decide this is the right way to go.

Comment by David Storch [ 15/Apr/20 ]

I'm unsure whether we want to go down the road of generally equating null and missing when it comes to the grammar of MQL. Generally speaking, an MQL expression like {$operator: {arg1: <value>, arg2: null}} is not currently considered identical in meaning to {$operator: {arg1: <value>}}. And when it comes to cleaning up null/missing semantics, my understanding is that the plan of record is to define that null != missing.

Comment by Charlie Swanson [ 14/Apr/20 ]

The query team decided that Jacob's proposal #2 is probably most user-friendly. Let's do that. Throwing this into a bucket of quick tech debt to hopefully schedule one BF Friday.

Comment by Jacob Evans [ 14/Apr/20 ]

The IDL treats null as missing, even going as far as to print error messages referring to them as such. Aggregate commands and $lookup stages do not accept null or missing, they accept only arrays, although they can be empty. So null == missing != [ ].

I propose continuing to equate null and missing. For the commands accepting arrays we should either:
1) Stay the course and allow only [ ] to mean nothing for pipeline arguments.
2) Allow pipelines to be optional/nothing in general and allow both null and missing.
Either approach would be consistent.

Comment by Charlie Swanson [ 02/Apr/20 ]

Hey jacob.evans can you dig in a little deeper to figure out a summary of how we treat null parameters? I remember some complexity in the mapReduce project where the drivers were sending null if the parameter was unspecified. I also somehow gained a belief that a field marked as "optional" in the IDL will accept null as equivalent to not present. I'm not entirely sure how I came to believe it or whether it's true... 

Another relevant question is how we treat the "pipeline" argument to $lookup and to the aggregate command in general. I suspect those will not accept null nor missing/unspecified.

Can you come up with one or more proposals of how we should proceed with optional parameters when it comes to null?

Generated at Thu Feb 08 05:12:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.