[CSHARP-3989] oDATA $select support with LINQ3 Created: 08/Dec/21  Updated: 28/Oct/23  Resolved: 01/Aug/23

Status: Closed
Project: C# Driver
Component/s: LINQ3, oData
Affects Version/s: 2.14.1
Fix Version/s: 2.21.0

Type: Bug Priority: Unknown
Reporter: Dimitri Kroo Assignee: Oleksandr Poliakov
Resolution: Fixed Votes: 4
Labels: LINQ
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on CSHARP-3494 Discriminator conventions don't work ... Backlog
depends on CSHARP-1423 System.ArgumentException in BsonMembe... Closed
depends on CSHARP-4118 Known serializers strategy is sometim... Closed
depends on CSHARP-1771 Support IIF method (i.e. ternary oper... Closed
Related
is related to CSHARP-4673 BsonSerializationException: Type Micr... Backlog
Epic Link: oData support for LINQ3
Quarter: FY24Q2
Case:
Backwards Compatibility: Fully Compatible
Documentation Changes: Not Needed
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Hello,

does a new LINQ3 support oDATA with $select?

I'm gettting following exception:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.InvalidOperationException: Cannot find serializer for value(Microsoft.AspNetCore.OData.Query.Container.LinqParameterContainer+TypedLinqParameterContainer`1[Microsoft.OData.Edm.IEdmModel]).
at MongoDB.Driver.Linq.Linq3Implementation.Serializers.KnownSerializers.KnownSerializersRegistry.GetSerializer(Expression expression, IBsonSerializer defaultSerializer)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ConstantExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, ConstantExpression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberExpression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberInitExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberInitExpression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.TranslateLambdaBody(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer, Boolean asRoot)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.SelectMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.TakeMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToExecutableQueryTranslators.ExpressionToExecutableQueryTranslator.Translate[TDocument,TOutput](MongoQueryProvider`1 provider, Expression expression)
at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.Execute()
at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.GetEnumerator()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at Microsoft.AspNetCore.OData.Query.Container.TruncatedCollection`1..ctor(IQueryable`1 source, Int32 pageSize, Boolean parameterize)
at Microsoft.AspNetCore.OData.Query.ODataQueryOptions.LimitResults[T](IQueryable`1 queryable, Int32 limit, Boolean parameterize, Boolean& resultsLimited)

Thank you!



 Comments   
Comment by Oleksandr Poliakov [ 01/Aug/23 ]

Some changes were done in Driver repository to improve oData support, but new package was created for more smooth integration and will be available soon:

https://github.com/mongodb/mongo-aspnetcore-odata

 

Comment by Robert Stam [ 21/Apr/22 ]

Over time this ticket has mentioned various different issues, some of which we have solved and then new ones appeared once execution got further along before running into an issue.

The latest attempt to reproduce issues mentioned in this ticket is here:

https://github.com/rstam/mongo-csharp-driver/tree/csharp3989-repro

Using the following query against the above branch:

http://localhost:5000/odata/products?$filter=Name%20eq%20%27Apple%27&$select=Name

generated the following LINQ query (captured using the debugger):

{test.products.Aggregate([])
    .Where($it => ($it.Name == value(Microsoft.AspNet.OData.Query.Expressions.LinqParameterContainer+TypedLinqParameterContainer`1[System.String]).TypedProperty))
    .Select($it => new SelectSome`1() {
        ModelID = value(Microsoft.AspNet.OData.Query.Expressions.LinqParameterContainer+TypedLinqParameterContainer`1[System.String]).TypedProperty, 
        Container = new NamedPropertyWithNext0`1() {
            Name = "Name"
            Value = IIF(($it == null), null, $it.Name), 
            Next0 = new AutoSelectedNamedProperty`1() {Name = "Id", Value = IIF(($it == null), null, Convert($it.Id, Nullable`1))}}})} 

which in turn got translated to the following MQL (captured using the debugger):

{ "$match" : { "name" : "Apple" } }
{ "$project" : { 
    "ModelID" : { "$let" : { "vars" : { "this" : { "TypedProperty" : "303f9f71-9098-4df0-9e74-25bb1d9c1e9f" } }, "in" : "$$this.TypedProperty" } }, 
    "Container" : { 
        "Name" : "Name"
        "Value" : { "$cond" : { "if" : { "$eq" : ["$$ROOT", null] }, "then" : null, "else" : "$name" } }, 
        "Next0" : { "Name" : "Id", "Value" : { "$cond" : { "if" : { "$eq" : ["$$ROOT", null] }, "then" : null, "else" : "$_id" } } }
    }, 
    "_id" : 0 }
} 

This pipeline did not error on the server, but an exception was thrown client side when trying to deserialize the result (captured using the debugger):

MemberAccessException: Cannot create an instance of Microsoft.AspNet.OData.Query.Expressions.PropertyContainer because it is an abstract class. 

The problem appears to originate in this part of the LINQ query:

Container = new NamedPropertyWithNext0`1() 

Container is declared as type `PropertyContainer` which is abstract. When deserializing "Container : { ... }" we think the type is `PropertyContainer` because that is the type of the `Container` property.

Most likely the generated MQL would need to insert an `_t : "new NamedPropertyWithNext0`1"` discriminator, but then the problem becomes that we don't yet support discriminators combined with generic types.

While this ticket has resulted in several minor issues being fixed, we have now reached a more difficult problem that we are not prepared to tackle immediately, so we are putting this ticket back in the backlog for now.

 

 

 

 

 

 

Comment by Robert Stam [ 07/Apr/22 ]

The primary issue I see above is one that we have removed from an OData context and minimally reproduced in CSHARP-4118.

There may be other LINQ queries that OData generates that we have trouble translating. We will have to deal with them as they arise and for each one the preferred course of action will be to create a non-OData related minimal reproduction and solve that.

Comment by John Youngers [ 24/Mar/22 ]

Thanks James!

For now I've wrapped the calls in a custom QueryProvider that will pull out the select portion of the expression, and execute that locally:

 

		public IQueryable<TElement> CreateMongoDbQueryable<TElement>(Expression expression)
		{
			// Sanitize Expression in case there's something in here MongoDb driver cannot handle
			var expressionVisitor = new MongoDbExpressionVisitor();
			var newExpression = expressionVisitor.Visit(expression)!;	
		
                        // If it was determined OData attempted to do projection ($select/$expand), handle that locally
			// Logic can be removed once this is resolved: https://jira.mongodb.org/browse/CSHARP-3989
			if (expressionVisitor.Select is { } odataSelect)
			{
				var elementType = newExpression.Type.GetGenericArguments()[0];
				var updatedQueryable = CreateNewMongoDbQueryable(_original, elementType, newExpression);				var enumerableResults = (IEnumerable)s_toArrayMethod.MakeGenericMethod(elementType)
					.Invoke(updatedQueryable, new object[] { updatedQueryable! })!;
				var queryableResults = enumerableResults.AsQueryable();
				var queryableWithSelect = queryableResults.Provider.CreateQuery(
					Expression.Call(
						typeof(Queryable),
						nameof(Queryable.Select),
						new[] { elementType, odataSelect.Body.Type },
						queryableResults.Expression,
						Expression.Quote(odataSelect)))!;
 
				return (IQueryable<TElement>)queryableWithSelect;
			}
			return _original.CreateQuery<TElement>(newExpression);
		}
 
		// MongoDbProvider does not implement the non-generic `CreateQuery` method, so we must do some voodoo to call the generic version
		private static IQueryable CreateNewMongoDbQueryable(IQueryProvider queryProvider, Type elementType, Expression expression)
		{
			s_mongoDbQueryProviderCreateQueryMethod ??= queryProvider.GetType()
				.GetRuntimeMethods()
				.Single(m => m.Name == nameof(IQueryProvider.CreateQuery) && m.IsGenericMethod);			return (IQueryable)s_mongoDbQueryProviderCreateQueryMethod.MakeGenericMethod(elementType)
				.Invoke(queryProvider, new object[] { expression })!;
		}

 

On a side note, in my use case OData would also include Converts that go from and to the same type, which was causing problems, so in the expression visitor I also have this entry:

 

		protected override Expression VisitUnary(UnaryExpression node)
		{
			// OData will create expressions like `Convert($it.Id, String)`, where $it.Id is already a string
			// Change it to simple be `$it.Id` as the MongoDb driver can't handle the expression 
			if (node.NodeType == ExpressionType.Convert && node.Type == node.Operand.Type)
			{
				return node.Operand;
			}			return base.VisitUnary(node);
		}

 

 

Comment by James Kovacs [ 24/Mar/22 ]

Thank you to everyone who has reached out for letting us know that oData and LINQ3 are still not playing well together. We are investigating further and will update this ticket. We appreciate your patience, but wanted to let you know that we are actively pursuing this issue.

Comment by Luca Vicenzotti [ 24/Mar/22 ]

Same issue here, $select statements in OData don't seem to work with MongoDB Linq provider V3....

Comment by beqa goderdzishvili [ 16/Mar/22 ]

What is the status of this issue?

I've tried both repositories: https://github.com/imallysson/mongodb-odata and https://github.com/marcosabotinski/mongodb-dotnetcore-odata-example (with latest dependencies for net6.0), but even simple $select query doesn't work... 

Comment by John Youngers [ 07/Mar/22 ]

I'm running into a similar issue; high level this is where the expression ends up:

 

MyDb.MyCollection.Aggregate([])
  .Select(Param_0 => new MyCollectionFacade() { Id = Param_0.Id,  Version = Param_0.Version})
  .OrderBy($it => $it.Version)
  .ThenBy($it => $it.Id)
  .Select($it => new SelectSome`1() { 
      Model = value(Microsoft.AspNetCore.OData.Query.Container.LinqParameterContainer+TypedLinqParameterContainer`1[Microsoft.OData.Edm.IEdmModel]).TypedProperty,
      Container = new NamedPropertyWithNext0`1() { 
          Name = "version"
          Value = Convert($it.Version, Nullable`1),  
          Next0 = new AutoSelectedNamedProperty`1() { 
                Name = "id"
                Value = $it.Id}}})

The issue is the `Model` property:  Is there a way to tell the driver if it hasn't found a Serializer, to just execute that part of the expression as is (as in it doesn't need to be a part of the actual query)?

Comment by Robert Stam [ 19/Feb/22 ]

allysson.lp@gmail.com thanks for the additional information. I'm currently working on other tickets but I appreciate very much that you've provided a repro.

Comment by Allysson Santos [ 18/Feb/22 ]

Hi @Robert Stam

I'm facing pretty much the same issue as @Dimitri Kroo.

I added to my github a project where you'll be able to reproduce the scenario.

 

GitHub: https://github.com/imallysson/mongodb-odata

OData query: https://localhost:44373/customer2?$select=id

Inside models folder you'll also find a small set of data I used.

 

Exception details:

System.InvalidOperationException: Cannot find serializer for value(Microsoft.AspNetCore.OData.Query.Container.LinqParameterContainer+TypedLinqParameterContainer`1[Microsoft.OData.Edm.IEdmModel]).
   at MongoDB.Driver.Linq.Linq3Implementation.Serializers.KnownSerializers.KnownSerializersRegistry.GetSerializer(Expression expression, IBsonSerializer defaultSerializer)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ConstantExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, ConstantExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberInitExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberInitExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.TranslateLambdaBody(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer, Boolean asRoot)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.SelectMethodToPipelineTranslator.Translate(TranslationContext context, MethodCallExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToPipelineTranslators.ExpressionToPipelineTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToExecutableQueryTranslators.ExpressionToExecutableQueryTranslator.Translate[TDocument,TOutput](MongoQueryProvider`1 provider, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.Execute()
   at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.GetEnumerator()
   at MongoDB.Driver.Linq.Linq3Implementation.MongoQuery`2.System.Collections.IEnumerable.GetEnumerator()
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value)
   at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Formatters.NewtonsoftJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

 

Comment by Dimitri Kroo [ 10/Jan/22 ]

Happy new year!

@Robert Stam
Can you please share your "simple OData query using LINQ3"? It should contain the list of objects with one property selected like https://url.com/odata/objects?$select=property.

And which version of Microsoft.AspNetCore.OData have you used?

Thanks!

Comment by Dimitri Kroo [ 20/Dec/21 ]

Thank you!
I'm using oMicrosoft.AspNetCore.OData 8.0.4.
The problem with $select is:

  • If it is only one object I do not get exception but I get all properties instead of provided in $select
  • If it is a list of objects I get an exception above

The class of the object(s) is very simple. Can you confirm that in your case $select works? If yes, I will try to create a repro project.

       var builder = new ODataConventionModelBuilder();
       builder.EntitySet<MyClass>("MyClass");

Comment by Robert Stam [ 18/Dec/21 ]

I have set up an OData environment and successfully executed a simple OData query using LINQ3 as the LINQ provider.

In order to investigate your scenario I will need more information, including:

  1. The model classes you are using
  2. The OData query URL that didn't work for you

Hopefully that will be enough for me to reproduce your scenario. If not, I'll follow up with you.

Thanks.

 

Generated at Wed Feb 07 21:46:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.