[CSHARP-2399] Truncation resulted in data loss. Created: 26/Sep/18  Updated: 03/Jan/19  Resolved: 03/Jan/19

Status: Closed
Project: C# Driver
Component/s: Serialization
Affects Version/s: 2.7.0
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Brecht Vanhaesebrouck Assignee: Robert Stam
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

.NET Core 2.1



 Description   

When running an aggregation through LINQ using IMongoQueryable I get this error when mapping to a decimal.

My mapping looks like this:

BsonClassMap.RegisterClassMap<SkuToOrder>(x =>
{
    x.AutoMap();
    x.MapProperty(prop => prop.CurrentStock).SetSerializer(new DecimalSerializer(BsonType.Decimal128, new RepresentationConverter(false, true)));
    x.MapProperty(prop => prop.NeededStock).SetSerializer(new DecimalSerializer(BsonType.Decimal128, new RepresentationConverter(false, true)));
    x.MapProperty(prop => prop.StockToOrder).SetSerializer(new DecimalSerializer(BsonType.Decimal128, new RepresentationConverter(false, true)));
    x.SetIgnoreExtraElements(true);
});

Here's an example model which generates this error:

{ 
    "Sku" : "0242", 
    "Description" : "xxx", 
    "CurrentStock" : NumberDecimal("0"), 
    "NeededStock" : NumberDecimal("0.6666666666666666666666666666666667"), 
    "StockToOrder" : NumberDecimal("0.6666666666666666666666666666666667")
}

 



 Comments   
Comment by Robert Stam [ 03/Jan/19 ]

I've made another attempt to reproduce this, but the only data I have to go on is the example model from your description. I inserted that model into a collection using the MongoDB shell and am able to read it into an `SkuToOrder` instance without an exception being thrown.

I can't actually run your LINQ query without some sample data.

I'm going to close this as Can't Reproduce for now. If you can provide an executable reproduction I will be happy to look at this again.

Comment by Robert Stam [ 03/Jan/19 ]

I will make another attempt to reproduce with the additional information you provided.

It would be really helpful if you could provide a complete standalone console application that reproduces this, including sample data. Either the console application could create its own test data, or a MongoDB shell script could create the sample data.

Comment by Brecht Vanhaesebrouck [ 15/Oct/18 ]

Here's the method I use to get the SKUs I need.

I look for the SKUs that have been ordered in the past time (in the orders collection).

Then I join the SKU collection to get the current stock.

Your SkuToOrder class is correct.

public IQueryable<SkuToOrder> GetSkusToOrder(DateTime baseDateFrom, DateTime baseDateTo, Int32 daysOfStockToKeep)
{
    baseDateFrom = new DateTime(baseDateFrom.Year, baseDateFrom.Month, baseDateFrom.Day, 0, 0, 1);
    baseDateTo = new DateTime(baseDateTo.Year, baseDateTo.Month, baseDateTo.Day, 23, 59, 59);
 
    var days = Math.Round((baseDateTo - baseDateFrom).TotalDays);
    var modifier = (Decimal)days / daysOfStockToKeep;
 
    var query = Database.GetCollection<Sku>("shop.orders")
        .Where(x => x.OrderDate >= baseDateFrom && x.OrderDate <= baseDateTo)
        .SelectMany(x => x.Skus)
        .GroupBy(x => new { x.Sku.Id, x.Sku.Description })
        .Select(x => new { Sku = x.Key.Id, NeededStock = x.Sum(s => s.Amount) / modifier })
        .Join(
            Database.GetCollection<Sku>("products.skus"),
            x => x.Sku,
            x => x.Id,
            (x, y) =>
                new SkuToOrder
                {
                    Sku = x.Sku,
                    Description = y.Description,
                    Supplier = y.Supplier,
                    CurrentStock = y.Stock,
                    NeededStock = x.NeededStock,
                    StockToOrder = x.NeededStock - y.Stock
                })
        .Where(x => x.StockToOrder > 0);
 
    return query;
}

Comment by Robert Stam [ 12/Oct/18 ]

I made a first attempt to reproduce using the following class definition for SkuToOrder

public class SkuToOrder
{
    public ObjectId Id { get; set; }
    public string Sku { get; set; }
    public string Description { get; set; }
    public Decimal CurrentStock { get; set; }
    public Decimal NeededStock { get; set; }
    public Decimal StockToOrder { get; set; }
}

I then inserted your sample document using the MongoDB Shell and was able to read it without error using either a Find or a LINQ expression

var client = new MongoClient("mongodb://localhost");
var database = client.GetDatabase("test");
var collection = database.GetCollection<SkuToOrder>("test");
 
foreach (var document in collection.Find("{}").ToEnumerable())
{
    var json = document.ToJson();
}
 
var result = collection.AsQueryable().Where(x => x.Sku == "0242").ToList();

So I'm assuming that in order to reproduce this I will need your actual LINQ expression.

Comment by Robert Stam [ 12/Oct/18 ]

In order to reproduce this can you please supply the definition of your SkuToOrder C# class and the LINQ expression that throws an exception?

Comment by Brecht Vanhaesebrouck [ 09/Oct/18 ]

I found a workaround for this issue:

Add an aditional aggregation step where you multiply the values by 100 and divide by 100 again.

Comment by Brecht Vanhaesebrouck [ 26/Sep/18 ]

Here's my stack trace:

MongoDB.Bson.Serialization.Options.RepresentationConverter.ToDecimal(Decimal128 value)
MongoDB.Bson.Serialization.Serializers.DecimalSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
MongoDB.Bson.Serialization.Serializers.SerializerBase<TValue>.MongoDB.Bson.Serialization.IBsoSerializer.Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
MongoDB.Bson.Serialization.IBsonSerializerExtensions.Deserialize(IBsonSerializer serializer, BsonDeserializationContext context)
MongoDB.Bson.Serialization.BsonClassMapSerializer<TClass>.DeserializeMemberValue(BsonDeseializationContext context, BsonMemberMap memberMap)

Generated at Wed Feb 07 21:42:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.