[CSHARP-4579] Pipeline projection translator does not use registered BsonClassMap and throws exception Created: 23/Mar/23  Updated: 31/Oct/23  Resolved: 25/Mar/23

Status: Closed
Project: C# Driver
Component/s: Linq, LINQ3
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Unknown
Reporter: x y Assignee: Oleksandr Poliakov
Resolution: Works as Designed Votes: 0
Labels: .net, c#
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to CSHARP-4819 ReplaceWith not respecting custom ele... Investigating
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

Summary

I have a class hierarchy with base and derived classes in C#.
The BsonId is stored on the base class and the derived classes hold some additional data.
I have an additional Id field on the derived class used for other purposes, and I set up the BsonClassMap to map it to “Id”, so it shouldn’t conflict with the BsonId.
I want to run a pipeline to make a query on the “derived” collection, but the pipeline fails to render to Bson.

The issue may be that ExpressionToAggregationExpressionTranslator creates a new BsonClassMap with AutoMap() and does not use the registered BsonClassMaps.

 

https://github.com/mongodb/mongo-csharp-driver/blob/master/src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToAggregationExpressionTranslators/MemberInitExpressionToAggregationExpressionTranslator.cs#L31

 

MongoDB.Driver version 2.19.0

How to Reproduce

Code:

using MongoDB.Bson;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
 
BsonClassMap.RegisterClassMap<Derived>(cm =>
{
	cm.AutoMap();
	cm.UnmapProperty(x => x.Id);
	cm.MapProperty(x => x.Id).SetElementName(nameof(Derived.Id));
}).Freeze();
 
var pipeline = new EmptyPipelineDefinition<Derived>()
	.Project(x => new Derived
	{
		Id = x.Id,
	});
 
var rendered = pipeline.Render(BsonSerializer.SerializerRegistry.GetSerializer<Derived>(), BsonSerializer.SerializerRegistry);
Console.WriteLine(rendered);
 
public abstract class Base
{
	[BsonId]
	[BsonElement("_id")]
	public ObjectId UniqueId { get; set; }
}
 
public class Derived : Base
{
	[BsonElement("Id")]
	public string Id { get; set; }
} 

Exception:

MongoDB.Bson.BsonSerializationException: The property 'Id' of type 'Derived' cannot use element name '_id' because it is already being used by property 'UniqueId' of type 'Base'.
   at MongoDB.Bson.Serialization.BsonClassMap.Freeze()
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.MemberInitExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, MemberInitExpression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.Translate(TranslationContext context, Expression expression)
   at MongoDB.Driver.Linq.Linq3Implementation.Translators.ExpressionToAggregationExpressionTranslators.ExpressionToAggregationExpressionTranslator.TranslateLambdaBody(TranslationContext context, LambdaExpression lambdaExpression, IBsonSerializer parameterSerializer, Boolean asRoot)
   at MongoDB.Driver.Linq.Linq3Implementation.LinqProviderAdapterV3.TranslateExpressionToProjection[TInput,TOutput](Expression`1 expression, IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, ExpressionTranslationOptions translationOptions)
   at MongoDB.Driver.ProjectExpressionProjection`2.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.PipelineStageDefinitionBuilder.<>c__DisplayClass39_0`2.<Project>b__0(IBsonSerializer`1 s, IBsonSerializerRegistry sr, LinqProvider linqProvider)
   at MongoDB.Driver.DelegatedPipelineStageDefinition`2.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.AppendedStagePipelineDefinition`3.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry, LinqProvider linqProvider)
   at MongoDB.Driver.PipelineDefinition`2.Render(IBsonSerializer`1 inputSerializer, IBsonSerializerRegistry serializerRegistry)
   at Program.<Main>$(String[] args) in C:\src\MongoTest\MongoTest\Program.cs:line 19
 

I tried it out with different property names that do not cause conflict, but I got strange results:
the translator does not use the BsonClassMap for the target property in the assignment, only for the source.

Code:

using MongoDB.Bson;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.Attributes;
using MongoDB.Driver;
 
BsonClassMap.RegisterClassMap<Derived>(cm =>
{
	cm.AutoMap();
	cm.UnmapProperty(x => x.Id2);
	cm.MapProperty(x => x.Id2).SetElementName("xyz");
}).Freeze();
 
var pipeline = new EmptyPipelineDefinition<Derived>()
	.Project(x => new Derived
	{
		Id2 = x.Id2,
	});
 
var rendered = pipeline.Render(BsonSerializer.SerializerRegistry.GetSerializer<Derived>(), BsonSerializer.SerializerRegistry);
foreach (var item in rendered.Documents)
	Console.WriteLine(item);
 
public abstract class Base
{
	[BsonId]
	[BsonElement("_id")]
	public ObjectId UniqueId { get; set; }
}
 
public class Derived : Base
{
	[BsonElement("abc")]
	public string Id2 { get; set; }
} 

Result:

{ "$project" : { "abc" : "$xyz", "_id" : 0 } } 

 



 Comments   
Comment by Robert Stam [ 27/Mar/23 ]

Interesting... I can't tell for sure whether you are only updating existing documents or adding new ones (depends on whether your are altering the UniqueId value or not in "other values").

If you are only updating existing documents you might want to consider using the overload of UpdateMany that applies a pipeline to each document that is to be updated. I believe that would be more efficient than using $merge back into the same collection.

We will consider whether it would be possible to use an existing registered class map (instead of automapping a new one) for you projection scenario. We would still have to overwrite the registered class map's serializers with serializers determined by the values flowing into the projection from the previous stage (working on a clone of the registered class map because the registered one is read-only). Most of the time the serializers flowing out from the previous stage would be equivalent to the serializers configured in the registered class map, but in case they don't match we have to use the serializer that matches the actual data coming from the previous stage.

As long as all the serializers  for each property in the pipeline projection match the serializers for the same properties in the registered class map you could then in principle $merge the documents back to the same collection. If they didn't match the new inserted or updated documents might have data serialized in a different representation.

Comment by x y [ 27/Mar/23 ]

My original code was about cloning some of the documents to the same collection with altering some of the properties.

I wanted to solve this by adding a merge to the end of the pipeline. In this example, I need the projection to keep the same property names from the original object, and any other settings from the registered BsonClassMap to match the schema of the collection.

 

 

        public static async Task CopyTo<T>(this IMongoCollection<T> collection, ObjectId sourceGroupId, ObjectId targetGroupId)
            where T : IEntity, new()
        {
            var pipeline = new EmptyPipelineDefinition<T>()
                .Match(x => x.GroupId == sourceGroupId)
                .Project(x => new T
                {
                    GroupId = targetGroupId,
                    Id = x.Id,
                    // ... other values
                })
                .Merge(collection, new MergeStageOptions<T>
                {
                    OnFieldNames = new[] { "_id" }
                });
            await collection.AggregateToCollectionAsync(pipeline);
        }

 

Comment by Robert Stam [ 27/Mar/23 ]

The reason we don't rely on any class maps that might have been registered is that there is no guarantee that the incoming data from the previous stage is serialized the same way as the members of the registered class map. They probably are, but there is no guarantee. So we build a new class map whose member serializers are derived from the data flowing into the projection.

I'm not sure I understand why the element names matter. As long as the driver correctly deserializes the results returned from the server the element names could be anything.

Comment by x y [ 25/Mar/23 ]

OK, this workaround could work for the Id property, but what about the second code example I provided? It's not just the Id property, it's about using the full dynamic power of BsonClassMap.

 

If I have a scenario, where property names are not known in advance when writing the code, only at runtime, how can I have the projection give correct results?

For example, when the program starts, it reads the mapping information from some data store, and registers BsonClassMaps based on that.

In this case I cannot add a BsonElement attribute in advance to the properties, and if I register a BsonClassMap, that wouldn't work for projection.

Is there a way to make the projection work correctly in this scenario?

 

 

Comment by Robert Stam [ 25/Mar/23 ]

While it is true that the LINQ3 translator does not use any registered class maps, it does use the class map automapping feature, which means that it will see and respect any attributes on the class declaration.

But you need to use a slightly different technique to suppress the automatic mapping of the `Id` property to the `_id` element name:

public abstract class Base
{
    [BsonId]
    [BsonElement("_id")] // this attribute is redundant given [BsonId]
    public int UniqueId \{ get; set; }
}
 
[BsonNoId]
public class Derived : Base
{
    public int Id \{ get; set; }
}

The trick is to use the `[BsonNoId]` attribute on the class, rather than `[BsonElement("Id")]` on the property itself.

You can see my repro (including a new scenario that also projects the `UniqueId` also) here:

https://github.com/rstam/mongo-csharp-driver/tree/csharp4579

Generated at Wed Feb 07 21:48:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.