[CSHARP-1894] InvalidCastException in FieldValueSerializerHelper on implicit type casting Created: 15/Jan/17  Updated: 08/Mar/17  Resolved: 13/Feb/17

Status: Closed
Project: C# Driver
Component/s: Linq, Serialization
Affects Version/s: 2.4.1
Fix Version/s: 2.4.3

Type: Bug Priority: Major - P3
Reporter: Vyacheslav Stroy Assignee: Robert Stam
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 10, MongoDB 3.4.0


Issue Links:
Depends
is depended on by CSHARP-1922 Filter with ReplaceOneAsync resuls in... Closed

 Description   

Recent driver release (v.2.4.1) has broken my project completely. It seems that the problem is related to the newly added serialization helper (FieldValueSerializerHelper). After updating from version 2.3.0 I got InvalidCastException.

Unable to cast object of type 'Fulcrum.Int64FulcrumId' to type 'Fulcrum.Ref`1[Fulcrum.TestData.ReferencedExampleEntity]'.
 
at MongoDB.Driver.FieldValueSerializerHelper.CastingSerializer`2.Serialize(BsonSerializationContext context, BsonSerializationArgs args, TFrom value)
   at MongoDB.Bson.Serialization.Serializers.SerializerBase`1.MongoDB.Bson.Serialization.IBsonSerializer.Serialize(BsonSerializationContext context, BsonSerializationArgs args, Object value)
   at MongoDB.Driver.Linq.Expressions.ISerializationExpressionExtensions.SerializeValues(ISerializationExpression field, Type itemType, IEnumerable values)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.TranslatePipelineContains(PipelineExpression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.TranslatePipeline(PipelineExpression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.Translate(Expression node, IBsonSerializerRegistry serializerRegistry)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateWhere(WhereExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateGroupBy(GroupByExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslatePipeline(PipelineExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node, IBsonSerializerRegistry serializerRegistry, ExpressionTranslationOptions translationOptions)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Translate(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Execute(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Execute[TResult](Expression expression)
   at System.Linq.Queryable.Count[TSource](IQueryable`1 source)

We are using active-record approach and lazy-load reference wrappers that can be casted to Id using implicit operator conversion.

var foo = new Foo();
foo.Save();
var bar = new Bar() { F = foo};
bar.Save();
 
//throws InvalidCastException
Bar.Collection.AsQueryable().Where(b=>b.F == foo.Id); 

FieldValueSerializerHelper.CastingSerializer is trying to convert value using boxing conversion. It will always fail if destination type does not match source type (the only exclusion is the situation when source type inherits from destination type).

_serializer.Serialize(context, args, (TTo)(object)value);

Boxing conversion omits user-defined type conversion therefor custom TypeConverter nor implicit operator conversions won't work.

Please add support for user-defined implicit type casting to the predicate translator.

I've prepared pull request that implements the requested feature. It might help with CSHARP-1890, CSHARP-1891 if I'm not mistaken. This approach will also allow implicit casting of primitive types out of the box, for example:

//Int32 (1) will be automatically casted to string value ("1")
SomeCollection<Example>().AsQueryable().Where(x=>x.StringValue == (object)1);
//{"StringValue": "1"}



 Comments   
Comment by Githook User [ 13/Feb/17 ]

Author:

{u'username': u'rstam', u'name': u'rstam', u'email': u'robert@robertstam.org'}

Message: CSHARP-1894: Code review changes.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/21e8768c964b37ed9f8b0039efdadff71fc465b3

Comment by Githook User [ 13/Feb/17 ]

Author:

{u'username': u'rstam', u'name': u'rstam', u'email': u'robert@robertstam.org'}

Message: CSHARP-1894: fallback to serializing as-is if value cannot be converted to field serializer.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/8fe8e0a2696a4255f0567a25842223d7be42ca27

Comment by Githook User [ 13/Feb/17 ]

Author:

{u'username': u'kreig', u'name': u'kreig', u'email': u'mrkreig@gmail.com'}

Message: CSHARP-1894 (InvalidCastException in FieldValueSerializerHelper on implicit type casting) fix updated to match current upstream master branch.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/6c7b2d5546b0516fff39ba1cfef9903e05b2347a

Comment by Vyacheslav Stroy [ 08/Feb/17 ]

I have updated my code. Please review that pull request again.
Thanks.

Comment by Vyacheslav Stroy [ 07/Feb/17 ]

Do you want to submit a new pull request? Your work looks good, but the existing pull request won't easily rebase on master.

Ok, I will update my pull request, no problem at all.

TypeConverters are primarily used in design-time environments to convert to and from strings.
I wonder how relevant they are here in the .NET driver to convert to and from arbitrary types.

Actually, TypeConverter approach is widely used in various low-level libraries. For example, I know for sure that TypeConverter technique is used by AutoMapper for unbound type casting. One of the main advantages over Convert.ChangeType() is that you can define custom type conversion for third-party types.

TypeDescriptor.AddAttributes(typeof(System.Array), new TypeConverterAttribute(typeof(MyCustomArrayConverter)));

In some cases it's even cheaper then Convert.ChangeType() call because specific type converter are usually designed to convert to/from particular types while Convert.ChangeType() has to perform multiple type checks. Convert performs better on primitive types (int, bool, long etc) because no boxing conversion applied for such calls as ToInt32().

Furthermore, Convert.ChangeType() will always throw exception on improper conversion attempt while TypeConverter provides CanConvertTo() and CanConvertFrom() methods that allow to check suitable types before conversion.

Comment by Robert Stam [ 07/Feb/17 ]

TypeConverters are primarily used in design-time environments to convert to and from strings.

I wonder how relevant they are here in the .NET driver to convert to and from arbitrary types.

Convert.ChangeType builds on top of IConvertible, which may or may not be more generally applicable.

Comment by Robert Stam [ 07/Feb/17 ]

Do you want to submit a new pull request? Your work looks good, but the existing pull request won't easily rebase on master.

I don't want to make extra work for you. If you prefer I can take your ideas and manually incorporate them into the current 2.4.2 code base.

Comment by Robert Stam [ 07/Feb/17 ]

Got it. I can reproduce this easily now also.

Thanks.

Comment by Vyacheslav Stroy [ 07/Feb/17 ]

"Foo and Bar" were given just as example in the original issue description.

The simplest way to reproduce:

//entity definition
public class Foo {
  public string StringProp {get;set;}
}
 
//query it given number as an argument
var value = (object)10; //wrap int to object
 
fooCollection.Where(foo => foo.StringProp == value).ToList(); //will throw InvalidCastException on v2.4.1

It's the first case, described actually in Implicit_Type_Casting_Primitive_Types test.

The user-specified conversion is described in Implicit_Type_Casting_With_Custom_TypeConverter test.
TypeConverterAttribute allows to add custom conversion to o from any type. So I created CExampleTypeCoverter that can be used to cast string to C class.

Comment by Robert Stam [ 07/Feb/17 ]

I had looked at the test case in the pull request and saw no mention of Foo and Bar.

Which classes in the new test case correspond to Foo and Bar?

Comment by Vyacheslav Stroy [ 07/Feb/17 ]

Yes, I wrote a test case. You can view it alongside with proposed solution in my pull-request.

public class PredicateImplicitTypeCastTests : IntegrationTestBase
{
    [Fact]
    public void Implicit_Type_Casting_Primitive_Types()
    {
        TypeDescriptor.AddAttributes(typeof(C), new TypeConverterAttribute(typeof(CExampleTypeCoverter)));
        Assert(
            x => x.B == (object)10,
            0,
            "{B: '10'}");
    }
 
    [Fact]
    public void Implicit_Type_Casting_With_Custom_TypeConverter()
    {
        //register custom type converter for C type
        TypeDescriptor.AddAttributes(typeof(C), new TypeConverterAttribute(typeof(CExampleTypeCoverter)));
        var objectToCast = (object)"Dexter";
        Assert(
            x => x.C == objectToCast,
            0,
            "{C: { '_t' : 'C', D: 'Dexter', E: null, S: null, X: null}}");
    }
 
    public void Assert(Expression<Func<Root, bool>> filter, int expectedCount, string expectedFilter)
    {
        var serializer = BsonSerializer.SerializerRegistry.GetSerializer<Root>();
        var filterDocument = PredicateTranslator.Translate(filter, serializer, BsonSerializer.SerializerRegistry);
 
        var list = __collection.FindSync(filterDocument).ToList();
 
        filterDocument.Should().Be(BsonDocument.Parse(expectedFilter));
        list.Count.Should().Be(expectedCount);
    }
 
    public class CExampleTypeCoverter : TypeConverter
    {
        public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
        {
            if (sourceType == typeof(string)) return true;
            return base.CanConvertFrom(context, sourceType);
        }
 
        public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
        {
            if (value is string) return CreateFromString(value);
            return base.ConvertFrom(context, culture, value);
        }
 
        public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
        {
#if NETSTANDARD1_5
                return (typeof(C).GetTypeInfo().IsAssignableFrom(destinationType));
#else
            return (typeof(C).IsAssignableFrom(destinationType));
#endif
        }
 
        public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
        {
#if NETSTANDARD1_5
                if (typeof(C).GetTypeInfo().IsAssignableFrom(destinationType))
#else
            if (typeof(C).IsAssignableFrom(destinationType))
#endif
            {
                if (value is string)
                {
                    return CreateFromString(value);
                }
            }
            return base.ConvertTo(context, culture, value, destinationType);
        }
 
        private C CreateFromString(object value)
        {
            return new C { D = (string)value };
        }
    }
}

Comment by Robert Stam [ 07/Feb/17 ]

Can you please provide the class definitions for Foo and Bar so that I may reproduce this locally?

Thanks.

Comment by Vyacheslav Stroy [ 07/Feb/17 ]

After updating to recent 2.4.2 release:

Object reference not set to an instance of an object.
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.Serialize(IBsonSerializer serializer, BsonSerializationContext context, Object value)
   at MongoDB.Bson.Serialization.IBsonSerializerExtensions.ToBsonValue(IBsonSerializer serializer, Object value)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.TranslateComparison(Expression variableExpression, ExpressionType operatorType, ConstantExpression constantExpression)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.TranslateComparison(BinaryExpression binaryExpression)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.TranslateAndAlso(BinaryExpression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.PredicateTranslator.Translate(Expression node, IBsonSerializerRegistry serializerRegistry)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateWhere(WhereExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslateGroupBy(GroupByExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.TranslatePipeline(PipelineExpression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node)
   at MongoDB.Driver.Linq.Translators.QueryableTranslator.Translate(Expression node, IBsonSerializerRegistry serializerRegistry, ExpressionTranslationOptions translationOptions)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Translate(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Execute(Expression expression)
   at MongoDB.Driver.Linq.MongoQueryProviderImpl`1.Execute[TResult](Expression expression)
   at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
   ...

Comment by Juliana Gradova [ 24/Jan/17 ]

I have the same issue with LINQ. Please add the custom serialization support as was mentioned by @VyacheslavStroy or simply turn off casting. CastingSerializer can be omitted to allow "as is" value serialization. In this case we can even specify custom serializer for specific type to control its db representation.

Comment by Vyacheslav Stroy [ 15/Jan/17 ]

Pull request

Generated at Wed Feb 07 21:40:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.