[CSHARP-1356] LINQ .Any(x => !list.Contains(x)) translates incorrectly. Created: 21/Jul/15  Updated: 16/Sep/19  Resolved: 29/Jan/19

Status: Closed
Project: C# Driver
Component/s: Linq
Affects Version/s: 2.0.1
Fix Version/s: 2.8.0

Type: Bug Priority: Major - P3
Reporter: Nicholas Markkula Assignee: Dmitry Lukyanov (Inactive)
Resolution: Done Votes: 0
Labels: None
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Attachments: Text File Program.cs    
Issue Links:
Related
related to CSHARP-2744 Consider to restore(or modify) the pr... Closed
is related to CSHARP-2012 != operator works incorrect for inter... Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
CSHARP-2591 LINQ - command ANY Sub-task Closed  

 Description   

I have a document with two arrays, one is a collection of string tags and the other is a collection of sub documents. I have created a mongo update that removes a sub document if any of the tags for the documents array are not in the provided list. When I execute the update I'm getting an IndexOutOfRangeException.

I've created a sample project to reproduce the bug which can be found here.
https://github.com/nickmkk/UpdateWhereItemsArentInProvidedList

Run the tests in RemoveSubEntityServiceTests.cs to reproduce the issue.

Note that the problem seems to be with the filter in RemoveSubEntityService.cs on line 25



 Comments   
Comment by Robert Stam [ 12/Sep/17 ]

I've attached a Program.cs to repro the latter case. It can be adapted to test the original case as well.

The program creates an array of documents and inserts them into a collection.

It then uses LINQ to Objects and LINQ to MongoDB to query the documents. The two should return the same set of documents but don't.

Finally, it manually creates the suggested correct translation and verifies that it also matches the same set of documents.

Comment by Robert Stam [ 12/Sep/17 ]

A similar scenario involving a top level negation generates an incorrect translation also:

public class Shipment
{
    public ObjectId Id { get; set; }
    [BsonElement("functions")]
    public string[] Functions { get; set; }
}
 
var myFunctions = new[] { "a", "b" };
var query = collection.Aggregate()
    .Match(s => !s.Functions.Any(f => !myFunctions.Contains(f)));
var operation = query.ToString();

The value of operation is:

"aggregate([{ \"$match\" : { \"functions\" : { \"$in\" : [\"a\", \"b\"] } } }])"

But that $match filter doesn't seem correct.

A correct translation of the filter might be:

{ functions : { $not : { $elemMatch : { $nin : [ "a", "b" ] } } } }

Comment by Nicholas Markkula [ 05/Apr/16 ]

We changed our data schema since then and our queries are much simpler now, no need to figure out a workaround anymore =). It was an interesting problem to try and solve though.

Thanks again

Comment by Craig Wilson [ 05/Apr/16 ]

Exactly. All isn't supported, and this is exactly the inverse. So, we really shouldn't be supporting it either. Just an accident that it is supported because

root.Tags.Any(x => providedTags.Contains(x))

is completely legal and code re-use has allowed the not to be included. I think the fix here is probably to simply stop supporting this.

As for a way to do this, you might need to break out the providedTags manually and use

$nor: [{"Tags": "tag1"}, {"Tags": "tag2"}]

I guess we could translate this internally as well. Anyways, we'll look into it.

Comment by Nicholas Markkula [ 05/Apr/16 ]

The linq expression is essentially the same as

root.Tags.All(x => providedTags.Contains(x))

which throws a not supported exception; I'm assuming that will be the solution with this expression as well. I was trying to find a work around to "All" not being supported, lol.

Thanks again for looking into it,

-Nick

Comment by Craig Wilson [ 05/Apr/16 ]

Thanks Nick.

Ok, this is interesting. We translate

root.Tags.Any(x => !providedTags.Contains(x))

into a $nin. So, in your example, the query is this:

{ "Tags" : { "$nin" : ["tag1"] } }

Which appears to be correct. However, MongoDB doesn't return the same results as Linq to Objects. I'm trying to figure out how you'd actually write this in MongoDB. I'm concerned that there may not be a way to do this. If that is true, we'll need to remove this translation from the query provider and start throwing an exception. I'll let you know what we find out.

Thanks for the report.

Comment by Nicholas Markkula [ 05/Apr/16 ]

Hey Craig,

Thanks for getting back to me, I just updated my example to mongo 2.2.3 and I'm still seeing issues. The index exception is gone but it still isn't updating entities that contain tags not in the provided tag list. See https://github.com/nickmkk/UpdateWhereItemsArentInProvidedList if you want to look at the working example. Run the tests and see that it fails to remove the sub document. I'm assuming this is just unsupported as I'm essentially doing an All filter.

Thanks for your time,

-Nick

Comment by Craig Wilson [ 05/Apr/16 ]

Hi Nicholas,

I'm so sorry. This ticket fell through the cracks. Are you still having this issue? I've been unable to reproduce this with the latest 2.2.3.

Craig

Generated at Wed Feb 07 21:39:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.