[CSHARP-4708] Support Expression.MakeIndex in LINQ3 Created: 06/Jul/23  Updated: 14/Jul/23  Resolved: 14/Jul/23

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: None
Fix Version/s: 2.21.0

Type: New Feature Priority: Unknown
Reporter: Pavel Levchuk Assignee: Robert Stam
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Documentation Changes: Not Needed
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Summary

By manually constructing Dictionary indexes with `Expression` class in Linq, I can't access dictionary keys with MongoDB LinqV3.

MongoDB.Driver.Linq.ExpressionNotSupportedException: Expression not supported: it.Fields.Item["DynamicField1{_}{_}"].

MongoDB Driver: 2.19.0

How to Reproduce

```

public class SampleDocument

{

    public string Id { get; set; }

    [BsonDictionaryOptions(DictionaryRepresentation.Document)]
    public required IDictionary<string, object?> Fields { get; set; }

}

 

public class SampleRepository

{

    private readonly IMongoCollection<SampleDocument> _sampleDocumentCollection;

  public async ValueTask LoadSampleDocuments(
        CancellationToken cancellationToken = default)
    {
        var baseQuery = _sampleDocumentCollection
            .AsQueryable();

        if (true) {

            __            

            var searchableFields = new[] { "DynamicField1", "DynamicField2" };

            var parameter = Expression.Parameter(typeof(SampleDocument), "it");

            var dictType = typeof(IDictionary<string, object>);
            var dictionaryIndexer = dictType.GetProperty("Item")!;

            var expressions = searchableFields
                .Select(Expression.Equal(
                        Expression.MakeIndex(
                            Expression.Property(parameter, nameof(PolicyMemberStateDocument.Fields)),
                            dictionaryIndexer,
                            new List<Expression> { Expression.Constant(fieldPath) }),
                        Expression.Constant("SearchValue")))
                .ToList();
            var dynamicFilter = expressions[0];

            if (expressions.Count > 1)
            {
                foreach (var expression in expressions.GetRange(1, expressions.Count - 1))
               

{                     dynamicFilter = Expression.Or(dynamicFilter, expression);                 }

            }

            var lambda = Expression.Lambda<Func<PolicyMemberStateDocument, bool>>(
                dynamicFilter,
                parameter);
            baseQuery = baseQuery.Where(lambda);

        }

       var result = await baseQuery.ToListAsync(cancellationToken);

}

```

Additional Background

Code in .NET repo that makes it `it.Fields.Item["DynamicField1"]` instead of `it.Fields["DynamicField1"]` : ExpressionStringBuilder.cs (microsoft.com)

I am building dynamic filtering for LinqV3 and encountered this issue for dynamic fields.

Stack trace:

MongoDB.Driver.Linq.ExpressionNotSupportedException: Expression not supported: it.Fields.Item["DynamicField1"].
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.BinaryExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, BinaryExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.BinaryExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, BinaryExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateUsingAggregationOperators(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.Translate(TranslationContext context, Expression expression, Boolean exprOk)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ExpressionToFilterTranslator.TranslateLambda(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer, Boolean asRoot)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.WhereMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.OrderByMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.GroupByMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.SelectMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.WhereMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.OrderByMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.SkipMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.TakeMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToExecutableQueryTranslators.ExpressionToExecutableQueryTranslator.Translate[TDocument,TOutput](MongoQueryProvider`1 provider, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.ToCursorAsync(CancellationToken cancellationToken)
   at MongoDB.Driver.IAsyncCursorSourceExtensions.ToListAsync[TDocument](IAsyncCursorSource`1 source, CancellationToken cancellationToken) 



 Comments   
Comment by Githook User [ 14/Jul/23 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-4708: Support Expression.MakeIndex in LINQ3.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/f0fdba6496edaddf4ed1b5f1635896a86080f2b7

Comment by Pavel Levchuk [ 13/Jul/23 ]

robert@mongodb.com Thanks for looking into it! Yep, that's how C# Expression.ToString() generates the expression code. Not sure why it doesn't generate it like get_item("key").

Comment by Robert Stam [ 11/Jul/23 ]

I've investigated further and determined that even though the Expression your code is using is not one that the C# compiler would generate, it nevertheless appears to be correct usage of `Expression.MakeIndex`.

Looks like we can support this new use case. Work is in progress.

Comment by Robert Stam [ 10/Jul/23 ]

The Expression we are expecting is: `it.Fields.get_Item("key")`

Comment by Robert Stam [ 10/Jul/23 ]

I don't think `it.Fields.Item["DynamicField1\{_}{_}"]` is a valid expression. It definitely doesn't compile using the C# compiler.

The correct Expression should involve a call go the property getter (which is called `get_Item`).

Do you have any further information about why that expression should be considered valid?

Comment by Pavel Levchuk [ 06/Jul/23 ]

Typo: PolicyMemberStateDocument should be SampleDocument in the code.

Comment by PM Bot [ 06/Jul/23 ]

Hi SpaierInstinct@gmail.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:49:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.