[CSHARP-4443] Support LINQ3 queries against all 3 possible representations of Dictionaries Created: 07/Dec/22  Updated: 30/Jun/23

Status: Backlog
Project: C# Driver
Component/s: LINQ3
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: Robert Stam Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CSHARP-4701 LINQ3 produces ExpressionNotSupported... Closed

 Description   

For example, a `Dictionary<string, int>` could be represented in 3 ways:

{ life : 41 }
[['life', 41]]
[{ k : 'life', v : 41 }]

Corresponding to `Documents`, `ArrayOfArrays` and `ArrayOfDocuments`.

We should support queries against all 3 representations.



 Comments   
Comment by Robert Stam [ 31/Mar/23 ]

I haven't had a chance to figure out exactly what your `OrderByDynamic` code is doing, but I can make the following two high level points:

1. If you are going to generate Expressions yourself you should generate exactly the same thing that the C# compiler would generate. You can write the desired expression in C# and run it in a debugger to see what the C# compiler generated.

2. LINQ queries against MongoDB can sometimes return subtly different results than the same query against Linq to Objects. The overall goal of Linq to MongoDB is to be an easy way to write MongoDB queries, not to return exactly the same results as Linq to Objects. When the results differ it will be because of subtle semantic differences between C# and MongoDB (for example, differing support for string comparisons using collations).

 

Comment by Manuel Kugelmann [ 31/Mar/23 ]

I guess there are subtle differences in the mapping of the LINQ expression between v2 and v3. 
The Dictionary indexer access can look like

var indexer = dictExpr.Type.GetProperty("Item");
var indexExpr = Expression.Property(dictExpr, indexer, keyExpr); 

or

var indexer = dictExpr.Type.GetProperty("Item"); 
var indexExpr = Expression.Call(dictExpr, indexer.GetGetMethod(), new[] { keyExpr });

or something else I didn't think of ...

Only the 2nd works with LINQ2. 
robert@mongodb.com How would a working Expression for LINQ3 look ? Do you have an hint where the relevant code is buried in the LINQ3 implementation to speed up my investigation?

This code block for namepath ( e.g. Blah.Blubb.Foo.Bar ) driven LINQ based filters works with  dictionaries as documents in v2 and not in v3:

        /// <summary>
        /// Order by a property, which name is given as a string.
        /// Inspired by https://stackoverflow.com/questions/36298868/how-to-dynamically-order-by-certain-entity-properties-in-entity-framework-7-cor
        /// Indexer access based on https://stackoverflow.com/questions/6759416/accessing-indexer-from-expression-tree
        /// </summary>
        private static IQueryable<T> OrderByDynamic<T>(IQueryable<T> source, string propertyName, bool ascending) {
            try {
                ParameterExpression parameter = Expression.Parameter(typeof(T), "x");
                var pathParts = propertyName.Split('.');
                Expression selector = Expression.PropertyOrField(parameter, pathParts[0]);
                for (int i = 1; i < pathParts.Length; i++) {
                    if (pathParts[i - 1] == "Extensions") { // TODO: Generalize for all Dictionaries
                        var dictExpr = selector;
                        var keyExpr = Expression.Constant(pathParts[i]);
                        PropertyInfo indexer = dictExpr.Type.GetProperty("Item")!;
                        var indexExpr = Expression.Call(dictExpr, indexer.GetGetMethod()!, new[] { keyExpr });
                        Type type = AppDomain.CurrentDomain.GetAssemblies()
                                   .SelectMany(x => x.GetTypes())
                                   .First(x => x.Name == pathParts[i]); // Key == TypeName for Extensions
                        selector = Expression.Convert(indexExpr, type);
                    } else {
                        selector = Expression.PropertyOrField(selector, pathParts[i]);
                    }
                }
                var method = ascending ? "OrderBy" : "OrderByDescending";
                var expression = Expression.Call(typeof(Queryable), method,
                    new Type[] { source.ElementType, selector.Type },
                    source.Expression, Expression.Quote(Expression.Lambda(selector, parameter)));
                var query = source.Provider.CreateQuery<T>(expression);
                return query;
            } catch (Exception ex) {
                Logging.Error(ex.Message); // TODO: add logging reference ?
                return source.AsQueryable<T>();
            }
        }
 

I know I could directly build a MongoDB filter, but our codebase above the DB access layer is DB agnostic because we target multiple DB implementations.
It is also nice for testing all higher logic against a LINQ-to-Object based DB Mock.

Comment by Robert Stam [ 29/Mar/23 ]

Thank you for letting us know you are having an issue with LINQ3. In general LINQ3 has way more functionality than LINQ2, but it is true that there is the occasional area where we have missed something in LINQ3, which we will of course address as we find out about it.

I believe that dictionaries represented as documents are already fully supported in LINQ3. This ticket is really just about the other two representations for dictionaries (array of documents and array of arrays).

When I get a chance I will confirm that dictionaries represented as documents are already supported.

Let us know if you are having any issues with dictionaries represented as documents.

Comment by Manuel Kugelmann [ 29/Mar/23 ]

Locked to LINQ V2 because of this.
Primarily need 'Documents' respresentation. Is it as easy to fix, as CSHARP-917?
Is there an ETA ?

V3 imho is NOT production ready if it has less functionality than V2 ... 

Generated at Wed Feb 07 21:48:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.