[CSHARP-4749] Unable to use user defined implicit conversion in filter Created: 09/Aug/23  Updated: 27/Oct/23  Resolved: 04/Sep/23

Status: Closed
Project: C# Driver
Component/s: LINQ3
Affects Version/s: 2.20.0
Fix Version/s: None

Type: Bug Priority: Unknown
Reporter: Aristarkh Zagorodnikov Assignee: Robert Stam
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi,

I mentioned in CSHARP-4681 that there were some incompatibilities with V3 driver and promised to create a ticket about that.

LinqV3 provider doesn't appear to respect implicit conversion operators for user defined types. Take a look at https://gist.github.com/onyxmaster/7139baa4059cb2648d15d3be683d089f for a repro (built against 2.20.0).



 Comments   
Comment by PM Bot [ 04/Sep/23 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to reopen this ticket if you're still experiencing the issue, and add a comment if you're able to provide more information.

Comment by PM Bot [ 25/Aug/23 ]

Hi onyxmaster! CSHARP-4749 is awaiting your response.

If this is still an issue for you, please open Jira to review the latest status and provide your feedback. Thanks!

Comment by Robert Stam [ 17/Aug/23 ]

Thanks for the additional information. Let us know if you find out anything else.

Comment by Aristarkh Zagorodnikov [ 10/Aug/23 ]

First, thank you, Robert, for the detailed explanation of the state of the things.

In our case, we have a lot of user-defined types like UserType, that imitate Haskell's/Rust's newtype and/or F#'s single-case discriminated union – that is, a wrapper around a (usually primitive) type that is used to discern, for example, UserAgeInYears (of int) from CarWeightInKgs (also of int).

These types always have a wrapped value, an implicit conversion operator to the type of this value (to "unwrap" them), a constructor that has this single value and a serializer (via serialization provider since they all implement a common interface IDomain<TValue>) which delegates serialization to child value serializers.

Given all that, the serialization for UserType(27383) in our case is NumberInt(27383), not { Value : NumberInt(27383) }.

With V2 of LINQ provider, it appears that an issue in its implementation worked in our favor and allowed us to use expressions like I described above. On the other hand, V3 appears to be stricter in this case, but since we still have the implicit conversion operator, the problem manifests itself in runtime, instead of compile-time. Removing the implicit conversion operator is a no-go for us, so if we're going to upgrade to V3 we've got to replace instances of Filter.XXX(d => d.Something, value) with the Filter.XXX(d => d.Something, new(value)) so the TField is of UserType, not the TValue contained inside. Since the implicit operators aren't going anywhere this might be less than trivial for us to do. Not impossible, of course, but will take some time for sure.

As a final note, I think there may be other people with similar issues, maybe the docs should include something to warn people against doing this. I'm not saying the docs are bad, in fact they are great, but since this is a nontrivial matter, maybe it deserves a sidenote or something.

Comment by Robert Stam [ 09/Aug/23 ]

Thanks for getting back to us.

Keep in mind the main point above: a LINQ provider cannot translate a user written conversion (implicit or explicit) because it cannot in general know what the conversion does.

Also keep in mind that the default serialization for {record struct UserType(int Value)} is { Value : 1 }, not 1.

In general (and by default) any class/struct/record in C# will become a document in MongoDB. If you want a class to be serialized in a non-default way you would have to write a custom serializer.

Comment by Aristarkh Zagorodnikov [ 09/Aug/23 ]

Robert, thanks for investigating the issue.

It looks like I was overzealous when trying to create a minimal repro and omitted some important code. I'm sorry for that, I didn't intend to create confusion with my report.

I will provide a better repro that will be closer to the real code, but maybe it will take some time since I've got a lot on my plate right now.

Comment by Robert Stam [ 09/Aug/23 ]

In this case it also looks like `UserType` is a good candidate for an `enum` instead of a `record`?

Comment by Robert Stam [ 09/Aug/23 ]

Note that if you define the implicit conversion in the other direction:

record struct UserType(int Value)
{
    //public static implicit operator int(UserType self) => self.Value;
    public static implicit operator UserType(int value) => new UserType(value);
}

it can work because then the implicit conversion is executed in your application before being passed to `Filter.Gt`.

Comment by Robert Stam [ 09/Aug/23 ]

The issue here is that you have a user defined implicit conversion (i.e. you wrote the code that does the conversion). LINQ providers in general can't know WHAT you did in your implicit conversion, and therefore cannot translate expressions that call user defined code to MQL.

The following code:

var filter = Builders<Document>.Filter.Gt(v => v.S, 1);

Is actually equivalent to the following code after the compiler infers the `<TField>` type argument and applies the implicit conversion to `int`:

var filter = Builders<Document>.Filter.Gt<int>(v => (int)v.S, 1);

This has to be translated to something that can be executed server side. Since the implicit conversion is user-written C# code (simple as it might be) there is no way for the LINQ provider to know what the implicit conversion actually does and therefore no way to translate that to something that can execute server side.

The only way you can get this query to work correctly is to not use the implicit conversion, like this:

var filter = Builders<Document>.Filter.Gt(v => v.S, new UserType(1));

You claim that this works in LINQ2, and that is not actually true. While it is true that LINQ2 does not throw an exception, the filter LINQ2 translates your original query to is:

{ S : { $gt : 1 } }

But that is not correct because the serialized representation of a `UserType` value is `

{ Value : 1 }

`, NOT the integer 1.

The correct translation of the filter is:

{ S : { $gt : { Value : 1 } } 

Comment by Robert Stam [ 09/Aug/23 ]

Thank you for reporting this. I will analyze the code you provided to see if I can determine what the actual issue and exception messages are and then update the ticket to more accurately describe this issue.

Comment by PM Bot [ 09/Aug/23 ]

Hi onyxmaster, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:49:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.