[CSHARP-4737] Linq3Implementation: MongoDB.Driver.Linq.ExpressionNotSupportedException Created: 01/Aug/23  Updated: 27/Oct/23  Resolved: 27/Oct/23

Status: Closed
Project: C# Driver
Component/s: Builders
Affects Version/s: 2.20.0
Fix Version/s: None

Type: Bug Priority: Blocker - P1
Reporter: Sasan Pasha Assignee: Robert Stam
Resolution: Won't Fix Votes: 1
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Summary

When i use update multiple fields, and the fields are multiple arrays, I get ExpressionNotSupportedException

MongoDB.Driver.Linq.ExpressionNotSupportedException: Expression not supported: b.Contributors.FirstMatchingElement().TransactionDetails.FirstOrDefault(t => (t.TransactionId == "b2231207-ad5c-4236-8517-5fdc5e7c5139")).
  at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ToFilterFieldTranslators.MethodCallExpressionToFilterFieldTranslator.Translate (MongoDB.Driver.Linq.Linq3Implementation.Translators.TranslationContext context, System.Linq.Expressions.MethodCallExpression expression) [0x0011a] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ToFilterFieldTranslators.ExpressionToFilterFieldTranslator.Translate (MongoDB.Driver.Linq.Linq3Implementation.Translators.TranslationContext context, System.Linq.Expressions.Expression expression) [0x00046] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ToFilterFieldTranslators.MemberExpressionToFilterFieldTranslator.Translate (MongoDB.Driver.Linq.Linq3Implementation.Translators.TranslationContext context, System.Linq.Expressions.MemberExpression memberExpression) [0x0000c] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToFilterTranslators.ToFilterFieldTranslators.ExpressionToFilterFieldTranslator.Translate (MongoDB.Driver.Linq.Linq3Implementation.Translators.TranslationContext context, System.Linq.Expressions.Expression expression) [0x00039] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.Linq.Linq3Implementation.LinqProviderAdapterV3.TranslateExpressionToField[TDocument,TField] (System.Linq.Expressions.Expression`1[TDelegate] expression, MongoDB.Bson.Serialization.IBsonSerializer`1[TValue] documentSerializer, MongoDB.Bson.Serialization.IBsonSerializerRegistry serializerRegistry, System.Boolean allowScalarValueForArrayField) [0x00037] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.ExpressionFieldDefinition`2[TDocument,TField].Render (MongoDB.Bson.Serialization.IBsonSerializer`1[TValue] documentSerializer, MongoDB.Bson.Serialization.IBsonSerializerRegistry serializerRegistry, MongoDB.Driver.Linq.LinqProvider linqProvider, System.Boolean allowScalarValueForArrayField) [0x00006] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.ExpressionFieldDefinition`2[TDocument,TField].Render (MongoDB.Bson.Serialization.IBsonSerializer`1[TValue] documentSerializer, MongoDB.Bson.Serialization.IBsonSerializerRegistry serializerRegistry, MongoDB.Driver.Linq.LinqProvider linqProvider) [0x00000] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.OperatorUpdateDefinition`2[TDocument,TField].Render (MongoDB.Bson.Serialization.IBsonSerializer`1[TValue] documentSerializer, MongoDB.Bson.Serialization.IBsonSerializerRegistry serializerRegistry, MongoDB.Driver.Linq.LinqProvider linqProvider) [0x00000] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.CombinedUpdateDefinition`1[TDocument].Render (MongoDB.Bson.Serialization.IBsonSerializer`1[TValue] documentSerializer, MongoDB.Bson.Serialization.IBsonSerializerRegistry serializerRegistry, MongoDB.Driver.Linq.LinqProvider linqProvider) [0x0001b] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.MongoCollectionImpl`1[TDocument].ConvertWriteModelToWriteRequest (MongoDB.Driver.WriteModel`1[TDocument] model, System.Int32 index) [0x0028b] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at System.Linq.Enumerable+<SelectIterator>d154`2[TSource,TResult].MoveNext () [0x0005a] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Linq/src/System/Linq/Select.cs:83 
  at System.Collections.Generic.List`1[T].AddEnumerable (System.Collections.Generic.IEnumerable`1[T] enumerable) [0x00059] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:1108 
  at System.Collections.Generic.List`1[T]..ctor (System.Collections.Generic.IEnumerable`1[T] collection) [0x00062] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/List.cs:87 
  at System.Linq.Enumerable.ToList[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x0000e] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/System.Linq/src/System/Linq/ToCollection.cs:30 
  at MongoDB.Driver.Core.Operations.BulkMixedWriteOperation..ctor (MongoDB.Driver.CollectionNamespace collectionNamespace, System.Collections.Generic.IEnumerable`1[T] requests, MongoDB.Driver.Core.WireProtocol.Messages.Encoders.MessageEncoderSettings messageEncoderSettings) [0x0000d] in <01310fd52db547f38c442be4cf246118>:0 
  at MongoDB.Driver.MongoCollectionImpl`1[TDocument].CreateBulkWriteOperation (MongoDB.Driver.IClientSessionHandle session, System.Collections.Generic.IEnumerable`1[T] requests, MongoDB.Driver.BulkWriteOptions options) [0x00033] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.MongoCollectionImpl`1[TDocument].BulkWriteAsync (MongoDB.Driver.IClientSessionHandle session, System.Collections.Generic.IEnumerable`1[T] requests, MongoDB.Driver.BulkWriteOptions options, System.Threading.CancellationToken cancellationToken) [0x0009e] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.MongoCollectionImpl`1[TDocument].UsingImplicitSessionAsync[TResult] (System.Func`2[T,TResult] funcAsync, System.Threading.CancellationToken cancellationToken) [0x000fe] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at MongoDB.Driver.MongoCollectionBase`1[TDocument].UpdateOneAsync (MongoDB.Driver.FilterDefinition`1[TDocument] filter, MongoDB.Driver.UpdateDefinition`1[TDocument] update, MongoDB.Driver.UpdateOptions options, System.Func`3[T1,T2,TResult] bulkWriteAsync) [0x00148] in <3907a4edc609468b9bf7e7be07687c1a>:0 
  at Infrastructure.Database.CosmoMongoDb.DeactivateReceiverFundingAsync (System.String fundingId, System.String allocationId) [0x00730] in C:\Dev\Repos\SensoDrive.Mobile\Infrastructure\Database\CosmoMongoDb.cs:1845 

 

Please provide the version of the driver. If applicable, please provide the MongoDB server version and topology (standalone, replica set, or sharded cluster).

How to Reproduce

Steps to reproduce. If possible, please include a Short, Self Contained, Correct (Compilable), Example.

               var builder = Builders<Funding>.Filter;
                var filter = builder.Eq(f => f.FundingId, fundingId) &
                             builder.ElemMatch(e => e.Receivers, r => r != null && r.AllocationId == receiver.AllocationId);
                var update = Builders<Funding>.Update.Set(b => b.Receivers.FirstMatchingElement().IsActive, false)
                    .Set(b => b.Receivers.FirstMatchingElement().IsDefault, false)
                    .Set(b => b.Receivers.FirstMatchingElement().ExpiryDateTime, DateTime.UtcNow)
                    .Set(b => b.Receivers.FirstMatchingElement().AllocatedAmount, receiver.UsedAmount)
                    .Set(b => b.Contributors.FirstMatchingElement().TransactionDetails.FirstOrDefault(t => t.TransactionId == receiver.FromTransactionId).AllocatedAmount, transaction.AllocatedAmount - credit);

                var result = await FundingsCollection.UpdateOneAsync(filter, update);

 

Additional Background

Please provide any additional background information that may be helpful in diagnosing the bug.



 Comments   
Comment by Robert Stam [ 27/Oct/23 ]

Closing for now as we have no plans at the moment to address this particular unusual edge case.

The root of the problem is that there is no equivalent MQL that the expression can be translated to.

Some users have commented that using array filters instead might work, but I have not attempted to verify that.

Comment by Shiya Kohn [ 24/Aug/23 ]

Hi sasan.pasha@gmail.com , there is no support for Linq and arrayFilters, and from my pedestrian review of the source code, there isn't really an easy path towards it. I think for now you have to use raw MQL for it

Comment by Sasan Pasha [ 24/Aug/23 ]

hi skohn@goflow.com  - I think arrayFilters is the way to go forward. however, there is not a good documentation on using c#, Linq and arrayfilters 

Comment by Shiya Kohn [ 24/Aug/23 ]

If you can think of some way to translate this to MQL that I am overlooking please let me know.

I don't think there's any easy way to do this in code, but what if it were able to become an arrayFilters?

As far as locally modifying the document and doing a replace, that can be risky and introduce race conditions.

Comment by Sasan Pasha [ 08/Aug/23 ]

thanks for the clarification - this is not effective coding. what about using array filters?

Comment by Robert Stam [ 08/Aug/23 ]

Thanks for reporting it even if it turns out that it can't be supported. It's always worth investigating.

There are limits to the kind of logic you can apply in an update command. Those limits are determined by what is possible to express in MQL.

You always have the option of fetching the entire document client side and modifying it locally (using the full power of C#), and then writing the document back to the server. The possible downside to this approach is that if the document is very large you are transferring a lot of data back and forth over the network.

Comment by Sasan Pasha [ 07/Aug/23 ]

hi robert@mongodb.com I don't have the expertise on the matter - I reported the issue since LINQ3 is not working for the query. Is there any alternative to run the above query with success? Or perhaps the community can translate that to MQL.

Comment by Robert Stam [ 07/Aug/23 ]

I don't think it is possible to translate this UpdateDefinition to MQL. There isn't any MQL that it can be translated to.

The issue is that the following expression needs to be translatable to a field name, but it is not possible:

b.Contributors.FirstMatchingElement().TransactionDetails.FirstOrDefault(t => t.TransactionId == receiver.FromTransactionId)

If you can think of some way to translate this to MQL that I am overlooking please let me know.

Comment by PM Bot [ 01/Aug/23 ]

Hi sasan.pasha@gmail.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:49:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.