[CSHARP-3614] LINQ translation sometimes uses wrong element name in projection Created: 23/Apr/21  Updated: 28/Oct/23  Resolved: 29/Jun/22

Status: Closed
Project: C# Driver
Component/s: Linq, LINQ3
Affects Version/s: 2.10.4
Fix Version/s: 2.17.0

Type: Bug Priority: Unknown
Reporter: Анатолий Крыжановский Assignee: Robert Stam
Resolution: Fixed Votes: 1
Labels: triaged
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Platform: .net core 3.1
Driver version: 2.10.4


Issue Links:
Duplicate
duplicates CSHARP-3235 FormatException when using AutoMapper... Closed
duplicates CSHARP-1771 Support IIF method (i.e. ternary oper... Closed
Epic Link: CSHARP-3615

 Description   

i have db model that contains all field of DTO for object.

For example, if we have entity Foo and it can be shown in table FooTableItemDto

 

FooTableItemDto
{
  public Guid Id { get; set; }
  public DateTime CreatedAt { get; set; }
  public string Creator { get; set; }
  public int TotalCost { get; set; }
}

as card FooViewDto:

 

 

FooViewDto
{
   public Guid Id { get; set; }
   public DateTime CreatedAt { get; set; }
   public string Creator { get; set; }
   public Guid CreatorId { get; set; }
   public FooMaterialDto Material { get; set; }
   public int TotalCost { get; set; }
}
 
FooMaterialDto
{
   public Guid Id { get; set; }
   public string MaterialName { get; set; }
   public Guid MaterialId { get; set; }
   public decimal Amount { get; set; }
   public decimal Price { get; set; }
}

in editor FooEditorDto

 

 

FooEditorDto
{
 public Guid Id { get; set; } 
 public FooMaterialDto Material { get; set; }
}

then we have all fields from this DTOs in db:

 

Foo
{
 public Guid Id { get; set; }
 public DateTime CreatedAt { get; set; }
 public string Creator { get; set; }
 public Guid CreatorId { get; set; }  
 public FooMateriall Material { get; set; }
 public int TotalCost { get; set; }
}
 
FooMateriall 
{
 public Guid Id { get; set; }
 public string MaterialName { get; set; }
 public Guid MaterialId { get; set; }
 public decimal Amount { get; set; }
 public decimal Price { get; set; }
}

 

 

now i have projector factory that build corresponding Expression. For example, if i want to get FooViewDto then factory generate me following expression:

 

(Foo x) => new FooViewDto
{
  Id = x.Id,
  CreatedAt = x.CreatedAt,
  Creator = x.Creator,
  CreatorId = x.CreatorId,
  Material = new FooMaterialDto
  {
     Id = x.Material.Id,
     MaterialName = x.Material.MaterialName,
     MaterialId = x.Material.MaterialId,
     Amount = x.Material.Amount,
     Price = x.Material.Price
  },
  TotalCost = x.TotalCost
}

 

 

and this works fine!

but now, i have case then Material can be null (user can create draft of document without filling this field), but my code does not failed with NullReferenceException but create empty object (with null or default values of all fields)

 

but i need that Material also be null in this case, so i add IIF expression - check if Material is null in db and return null in such case, otherwise create projection:

 

(Foo x) => new FooViewDto
{
 Id = x.Id,
 CreatedAt = x.CreatedAt,
 Creator = x.Creator,
 CreatorId = x.CreatorId,
 Material = IIF(x.Material == null), null, new FooMaterialDto
 {
   Id = x.Material.Id,
   MaterialName = x.Material.MaterialName,
   MaterialId = x.Material.MaterialId,
   Amount = x.Material.Amount,
   Price = x.Material.Price
 },
 TotalCost = x.TotalCost
}

 

but in this case i got exception during projection:

Element 'Id' does not match any field or property of class FooMaterialDto

 

the exception was gone if i add explicit mapping for FooMaterialDto. But i don't want to do that, because i suggest that must have mapping only for my db entities, not for dto. and i don't have such things in case if i don't have IIF operation in my expression

so my question is - what i do wrong or missed?



 Comments   
Comment by Robert Stam [ 29/Jun/22 ]

This issue has been fixed in the new LINQ provider (known as LINQ3), which is included in the 2.14 release.

Configure your MongoClientSettings to use LinqProvider.V3 if you want to use this functionality.

To configure a client to use the LINQ3 provider use code like the following

var connectionString = "mongodb://localhost";
var clientSettings = MongoClientSettings.FromConnectionString(connectionString);
clientSettings.LinqProvider = LinqProvider.V3;
var client = new MongoClient(clientSettings);

Comment by Githook User [ 29/Jun/22 ]

Author:

{'name': 'rstam', 'email': 'robert@robertstam.org', 'username': 'rstam'}

Message: CSHARP-3614: Verify that issue is not present in LINQ3.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/c66a4e60e2f6b5e68d3d4686aea1b3be91438755

Comment by Robert Stam [ 17/May/21 ]

This looks like a bug in the LINQ translation of the query.

The current implementation sends the following pipeline to the server:

{ "$project" : { 
    "Id" : "$_id", 
    "PageCount" : "$PageCount", 
    "Author" : { "$cond" : [{ "$eq" : ["$Author", null] }, null, { "Id" : "$Author._id", "Name" : "$Author.Name" }] },
     "_id" : 0 } }

 
But this is incorrect because the serialized forms of BookDto and AuthorDto expect `_id` instead of `Id`.

The pipeline should be:

{ "$project" : { 
    "_id" : "$_id", 
    "PageCount" : "$PageCount", 
    "Author" : { "$cond" : [{ "$eq" : ["$Author", null] }, null, { "_id" : "$Author._id", "Name" : "$Author.Name" }] } }

We are currently reimplementing our LINQ provider and we will ensure that this use case works in the new LINQ provider.

I can offer two workarounds you could use in the meantime:

1. Annotate your BookDto and AuthorDto with the [BsonNoId] attribute (this disables the mapping of "Id" to "_id" as the element name)
2. Rename the Id properties in your BookDto and AuthorDto to "BookId" and "AuthorId" (not using the name "Id" avoids the issue)

Comment by Анатолий Крыжановский [ 14/May/21 ]

any updates?

Comment by Анатолий Крыжановский [ 28/Apr/21 ]

so, i created sample which illustrate problem. we have two entity, Book and they Author (Author info is incapsulated in Book), also we have dto's for this entities. 

Then we try to select all books with author info. 

There are two query - simple, which project one-to-one. This query works but return empty Author info instead of null

Second query contains IIF operation and thro expection

 

using System;
using System.Linq;
using MongoDB.Bson;
using MongoDB.Bson.Serialization;
using MongoDB.Bson.Serialization.IdGenerators;
using MongoDB.Driver;
using MongoDemo.Dto;
using MongoDemo.Mapping;
using MongoDemo.Models;
 
namespace MongoDemo.Dto
{
   public class BookDto
   {
      public Guid Id { get; set; }
      public int PageCount { get; set; }
      public AuthorDto Author { get; set; }
   }
   
   public class AuthorDto
   {
      public Guid Id { get; set; }
      public string Name { get; set; }
   }
}
 
namespace MongoDemo.Models
{
   public class Author : IEntity
   {
      public Guid Id { get; set; }
      public string Name { get; set; }
   }
   
   public class Book: IEntity
   {
      public Guid Id { get; set; }
      public int PageCount { get; set; }
      public Author Author { get; set; }
   }
   
   public interface IEntity
   {
      Guid Id { get; set; }
   }
}
 
namespace MongoDemo.Mapping
{
   public class AuthorMapping: BsonClassMap<Author>
   {
      public AuthorMapping()
      {
         MapIdProperty(x => x.Id)
            .SetIdGenerator(new NullIdChecker());
         
         MapProperty(x => x.Name);
      }
   }
   
   public class BookMapping: BsonClassMap<Book>
   {
      public BookMapping()
      {
         MapIdProperty(x => x.Id)
            .SetIdGenerator(new NullIdChecker());
         
         MapProperty(x => x.PageCount);
         MapProperty(x => x.Author);
      }
   }
}
 
namespace MongoDemo
{
   
   
   
   
   class Program
   {
      private static MongoClient _dbClient;
      private static IMongoDatabase _database;
 
      static void Main(string[] args)
      {
         CreateMapping();
         Connect();
         CreateData();
         ExecuteSimpleQuery();
         ExecuteIIFQuery();
      }
 
      static void ExecuteIIFQuery()
      {
         Console.Write("Execute IIF projection query");
         var data = _database
            .GetCollection<Book>("books")
            .AsQueryable()
            .Select(x => new BookDto
            {
               Id = x.Id,
               PageCount = x.PageCount,
               Author = x.Author == null
                  ? null
                  : new AuthorDto
                  {
                     Id = x.Author.Id,
                     Name = x.Author.Name
                  }
            })
            .ToArray();
         
         Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(data));
      }
      
      static void ExecuteSimpleQuery()
      {
         Console.Write("Execute simple projection query");
         var data = _database
            .GetCollection<Book>("books")
            .AsQueryable()
            .Select(x => new BookDto
            {
               Id = x.Id,
               PageCount = x.PageCount,
               Author = new AuthorDto
               {
                  Id = x.Author.Id,
                  Name = x.Author.Name
               }
            })
            .ToArray();
         
         Console.WriteLine(System.Text.Json.JsonSerializer.Serialize(data));
      }
 
      static void CreateData()
      {
         var book1Id = Guid.Parse("52f7b6ba-a893-48f2-afaf-91c3102d7e81");
         var book2Id = Guid.Parse("52f7b6ba-a893-48f2-afaf-91c3102d7e82");
         var author1Id = Guid.Parse("52f7b6ba-a893-48f2-afaf-91c3102d7e83");
 
         var book1 = new Book
         {
            Id = book1Id,
            PageCount = 1,
            Author = new Author
            {
               Id = author1Id,
               Name = "author1"
            }
         };
         
         var book2 = new Book
         {
            Id = book2Id,
            PageCount = 2,
            Author = null
         };
 
         var collection = _database.GetCollection<Book>("books");
         var options = new ReplaceOptions
         {
            IsUpsert = true
         };
 
         collection.ReplaceOne(x => x.Id == book1Id, book1, options);
         collection.ReplaceOne(x => x.Id == book2Id, book2, options);
      }
 
      static void Connect()
      {
         _dbClient = new MongoClient(MongoClientSettings.FromConnectionString("mongodb://localhost/"));
         _database = _dbClient.GetDatabase("data", (MongoDatabaseSettings) null);
      }
 
 
      static void CreateMapping()
      {
         BsonDefaults.GuidRepresentation = GuidRepresentation.Standard;
         
         BsonClassMap.RegisterClassMap(new BookMapping());
         BsonClassMap.RegisterClassMap(new AuthorMapping());
      }
   }
}

Comment by Анатолий Крыжановский [ 27/Apr/21 ]

Ok, i try to create sample tommorow

Comment by Mikalai Mazurenka (Inactive) [ 23/Apr/21 ]

Hello anatoly.kryzhanovsky@singularis-lab.com,

Thank you for reporting this issue.

This case looks like CSHARP-1771, but to confirm it we will need you to provide a self-contained reproduction.

Note, that we are currently reimplementing our LINQ translation layer, and considering a fix for the linked issue.

Comment by Анатолий Крыжановский [ 23/Apr/21 ]

and more info

i found that there is different behavior for IntiMember Exrepssion and other type

if we have simple InitMember then we create BsonMap without id member and all works fine

in my case i have IIF expression and AutoClass was invoked, during autoclasss NamedIdMemberConvention executed

 

so as workaround i remove _defaults_ convention pack and add it with type filter (apply only for my db models)

also i create new conventionpack which does not include NamedIdMemberConvention and apply them with type filter (apply for all other types)

Comment by Анатолий Крыжановский [ 23/Apr/21 ]

additional info:

i go deeper while debugging and found that there are difference between BsonTrie if i have IIF operation and without it

for IIF expression case i have property "_id" (and i don't have such property in my DTO, and otherwise - in my Dto i have Id but there is no such property in BsonTrie)

for non-IIF expression i have "Id" property

Generated at Wed Feb 07 21:45:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.