[CSHARP-3181] Default DateTimeOffset serialization makes it impossible to search across timezones Created: 12/Aug/20  Updated: 09/Jan/24

Status: Backlog
Project: C# Driver
Component/s: Serialization
Affects Version/s: None
Fix Version/s: 3.0.0

Type: Bug Priority: Major - P3
Reporter: Diego Frata Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: next-major
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: Implement 3.0 release
Quarter: FY24Q3
Case:
Documentation Changes Summary:

1. What would you like to communicate to the user about this feature?
2. Would you like the user to see examples of the syntax and/or executable code and its output?
3. Which versions of the driver/connector does this apply to?


 Description   

The default serialization for DateTimeOffset persists two values: a local date time (in ticks) + the original timezone, preventing searching across timezones.  Consider the example:

Document A has the timestamp '2020-01-10 10:00:00 +00:00' and document B has the timestamp '2020-01-10 11:00:00 +01:00'. In .NET, comparing both timestamps returns true, because even though their timezones are different, both are referring to the same point in time.

 

Now, if I persist both documents to MongoDB and later I search for these documents using the timestamp '2020-01-10 10:00:00 +00:00', only the first document will be returned. If I search for '2020-01-10 11:00:00 +01:00' then only the second document will be returned. This is not the expected behaviour, essentially the search only works on local times and it even disregards the timezone.

 

Someone else raised this issue before on CSHARP-2603, but it was dismissed as support and no follow up is recorded. I solved this problem in my codebase by rolling my own serializer, which serialized the DateTime part of the type as UtcTicks, essentially solving the problem.

 

public class UtcDateTimeOffsetSerializer : StructSerializerBase<DateTimeOffset>
{
    // public methods
    /// <summary>
    /// Deserializes a value.
    /// </summary>
    /// <param name="context">The deserialization context.</param>
    /// <param name="args">The deserialization args.</param>
    /// <returns>A deserialized value.</returns>
    public override DateTimeOffset Deserialize(BsonDeserializationContext context, BsonDeserializationArgs args)
    {
        var bsonReader = context.Reader;
        var bsonType = bsonReader.GetCurrentBsonType();
        
        if (bsonType != BsonType.Array)
            throw CreateCannotDeserializeFromBsonTypeException(bsonType);
 
        bsonReader.ReadStartArray();
        var ticks = bsonReader.ReadInt64();
        var offset = TimeSpan.FromMinutes(bsonReader.ReadInt32());
        bsonReader.ReadEndArray();
        return new DateTimeOffset(ticks + offset.Ticks, offset);
    }
 
    /// <summary>
    /// Serializes a value.
    /// </summary>
    /// <param name="context">The serialization context.</param>
    /// <param name="args">The serialization args.</param>
    /// <param name="value">The object.</param>
    public override void Serialize(BsonSerializationContext context, BsonSerializationArgs args, DateTimeOffset value)
    {
        var bsonWriter = context.Writer;
        bsonWriter.WriteStartArray();
        bsonWriter.WriteInt64(value.UtcTicks);
        bsonWriter.WriteInt32((int) value.Offset.TotalMinutes);
        bsonWriter.WriteEndArray();
    }
}
 

 

Given that this format is essentially incompatible with existing data, I would suggest that the default representation is changed from Array to Document so the DateTime portion is recorded in UTC epoch milliseconds, which is at least searchable.



 Comments   
Comment by Esha Bhargava [ 28/Sep/20 ]

d@frata.io Thanks again for reporting this! We'll consider this for the next major release.

Comment by Diego Frata [ 29/Aug/20 ]

I agree it’s a breaking change and should go as part of a major release. It is impossible to know, however I would be surprised anyone doing production work with DateTimeOffset is actually using the default format.

Comment by Robert Stam [ 29/Aug/20 ]

I agree with you that the Document representation would be a better default.

Unfortunately, changing the default representation would be a backward breaking change so I'm not sure we can do that.

Comment by Diego Frata [ 29/Aug/20 ]

I am aware that the Document mode works, but it’s really painful that the default serialisation mode is Array, which doesn’t work at all for queries involving different timezones, which is exactly the point of using DateTimeOffset. I have seen two teams inside my company already being bitten by this issue.

As the outcome of this ticket, I’d hope that the default serialisation mode is changed to document in newer versions of the driver, this would save other devs a lot of headache in migrating existing data to the document format. Do you agree with my proposal?

Comment by Robert Stam [ 28/Aug/20 ]

Have you tried the option of serializing DateTimeOffset as a document instead of the default array?

The resulting serialized format would look like:

{ "DateTime" : ISODate("2020-08-28T23:49:02.222Z"), "Ticks" : NumberLong("637342301422226358"), "Offset" : -420 }

The DateTime component is stored as a BsonDate (converted to UTC) which should correctly support all of the following operations on the server:

  • Comparing moments in time across time zones (compare the DateTime component)
  • Indexing on moments in time across time zones (index on the DateTime component)
  • Sorting on moments in time across time zones (sort on the DateTime component)

In this representation DateTimeOffset values that represent the same moment in time in different time zones will have the same value for the DateTime field but will have the local time zone information in the Ticks and Offset fields.

You can either configure serialization of your DateTimeOffset properties on a class by class basis or you can register an application-wide default serializer for DateTimeOffset by putting the following line of code in your application startup path:

BsonSerializer.RegisterSerializer(new DateTimeOffsetSerializer(BsonType.Document));

Generated at Wed Feb 07 21:44:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.