[SERVER-1454] Use $not as a top-level logical op Created: 21/Jul/10  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Trivial - P5
Reporter: Steve Wagner Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 17
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File top_level_not.patch    
Issue Links:
Backports
Duplicate
is duplicated by SERVER-10708 negation of query expressions using "... Closed
Related
related to SERVER-39019 $elemMatch $ne serialization is incor... Closed
related to SERVER-43349 Incorrect serialization for $elemMatc... Closed
related to JAVA-4008 Document JAVA-4006 in Driver Upgrade ... Closed
is related to JAVA-4006 Filters can produce invalid query sha... Closed
is related to SERVER-58772 improve error message for top level $... Closed
Assigned Teams:
Query Optimization
Backport Requested:
v4.4, v4.2, v4.0
Sprint: Query Optimization 2021-06-14, QO 2021-09-20, QO 2021-10-04, QO 2021-10-18, QO 2021-11-01, QO 2021-11-15, QO 2021-11-29, QO 2021-12-13, QO 2021-12-27, QO 2022-01-10, QO 2022-01-24, QO 2022-02-07, QO 2022-02-21, QO 2022-03-07
Participants:
Case:

 Description   

In our MongoDB-CSharp driver we convert code expressions like that to documents queries.

!(Address.IsDefault && Adress.Name=="foo")

The problem here is that the ! operator reverses the meaning of both equality checks. This scenario would be much easier to accomplish when we could use the $not operator like the $or operator.

Example:

{ $not : [ 
    { "Address.IsDefault" : true },
    { "Address.Name" : "foo" }
] }

I think also that this is more the way the users expect to use the $not operator.



 Comments   
Comment by Steve La (Inactive) [ 11/Dec/21 ]

Hi Joe
I understand from Jacob this is not something we can do until we support
CST ( new parser for our optimizer ), that likely won't come until late
next year
I suggest we just file this back into our general backlog instead of moving
it from sprint to sprint

Thx
Steve

Comment by Timour Katchaounov [ 11/Mar/21 ]

A WIP code review.

Comment by Charlie Swanson [ 24/Feb/21 ]

I uploaded top_level_not.patch as a place to start here. It's not well tested but it proves that it shouldn't be crazy difficult. One thing I noticed that we'd probably want to fix here: the way we serialize NOT is pretty backwards right now. Our code parses something like 

{x: {$not: {$gt: 4}}}

into a MatchExpression tree with NOT -> GT(x, 4). Then it has to extract the "x" and serialize back to the original. So there's some finicky code to try to search within a $not sub-tree to find the path. All of that can probably be removed if we allow $not to be top-level. I would proceed with caution here because it's been a source of some bugs. I'm not sure why we didn't just allow this syntax for SERVER-43349 and SERVER-39019.

It could be related to upgrade/downgrade concerns. If we start serializing in a new format, we need to make sure everyone in the cluster understands that format. For mongos this shouldn't be a problem because mongos is the last to be upgraded, but we do send queries from shard to shard for cases like $merge. In these cases we'll need to make sure we're on the upgraded FCV before using the new serialization format - which would cause us to keep around the old serialization code, at least for one major release.
 

Comment by Charlie Swanson [ 09/Feb/21 ]

I'm nominating this as quick win because I have no reason to suspect it's all that hard and it would simplify some of the language building efforts on the drivers side.

Comment by Asya Kamsky [ 15/Apr/16 ]

Equivalent of top level $not that already works now would be {$nor: [ { original-query } ] }

Comment by Eugen M [ 17/Dec/15 ]

To Robert Stam:
Yes. Transformation should support the law of de Morgan . Which is not ambiguous because it law )

For MongoDB-developers, I think, the ambiguity or complexity would not be at all. Since they will only take those objects that do not fall under a set of conditions shown in $ not: {}.

Comment by Eugen M [ 17/Dec/15 ]

it will be a very good feature!
For example: i take conditions from admins, in wich case they dont see documents. And witout this feature i create in aggreagte.project() FOR ALL DOCUMENTS computed fields with false to prohibited fields and THEN match all documents for computed field == true.
It works very slowly! (

PS in my example i need to ADD ALL FIELDS in project! Automotically it imposible and i use "fields: "$$ROOT"", then i need to exclude this new level (named "fields").... This is unnormal way! (

Comment by Robert Stam [ 07/Apr/15 ]

I think $not should take a single argument, otherwise it can be rather ambiguous what it means (does De Morgan's Law apply or not?).

The example in the description could be represented unambiguously as:

{ $not : 
    { $and : [
        { "Address.IsDefault" : true },
        { "Address.Name" : "foo" }
    ] } 
}

Comment by Nathan Ehresman [ 30/Dec/10 ]

I have a situation where expanded $not support would be very useful. We have a query builder to let the user build complex expressions that are then translated into a MongoDB query and executed.

Comment by Eliot Horowitz (Inactive) [ 21/Jul/10 ]

When we prioritize this will depend on # of votes primarily.

Comment by Steve Wagner [ 21/Jul/10 ]

I would really like to know if you consider adding this or not soon.

Generated at Thu Feb 08 02:57:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.