[CSHARP-2233] BsonDocument.Parse fails to recognize ISO 8601 dates Created: 02/Apr/18  Updated: 27/Oct/23  Resolved: 02/Apr/18

Status: Closed
Project: C# Driver
Component/s: Json
Affects Version/s: 2.5
Fix Version/s: None

Type: Task Priority: Minor - P4
Reporter: Hugh Williams Assignee: Robert Stam
Resolution: Works as Designed Votes: 0
Labels: question
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

I need to take a JSON string that came out of JSON.NET and read it into a BsonDocument.

Expected: the BsonDocument will contain an ISODate
Observed: the BsonDocument treats the date as a string

Here's a quick repro that illustrates what I'm seeing:

public void SerializeDates()
{
    var now = DateTime.Parse("2018-04-01T23:43:24.1234567-04:00", CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal);
    var obj = new {now};
    var json = Newtonsoft.Json.JsonConvert.SerializeObject(obj);
    var bsonFromJson = BsonDocument.Parse(json);
    var bson = obj.ToBsonDocument();
    Console.WriteLine("JSON          : {0}", json);          // prints {"now":"2018-04-02T03:43:24.1234567Z"}
    Console.WriteLine("BSON from JSON: {0}", bsonFromJson);  // prints { "now" : "2018-04-02T03:43:24.1234567Z" }
    Console.WriteLine("BSON          : {0}", bson);          // prints { "now" : ISODate("2018-04-02T03:43:24.123Z") }
}



 Comments   
Comment by Hugh Williams [ 02/Apr/18 ]

Thanks, that makes sense.

If anyone else finds it useful, the workaround code I'm going forward with is shared at https://gist.github.com/hughbiquitous/d1098da73767a6bbd5889bbb8c3dd1a8

Comment by Robert Stam [ 02/Apr/18 ]

Thank you for contacting us about this.

The underlying issue is that JSON does not define a representation for DateTimes. Json.NET and MongoDB have solved this limitation in different ways.

As you know, Json.NET decided to represent DateTimes using strings. But that makes reading strings ambiguous (is it really a string or is it really a DateTime?).

MongoDB decided to represent DateTimes (and other BSON data types that JSON does not support) using what is called "Extended JSON". The C# driver also uses an easier to read representation called "Shell" representation that can be copy/pasted into the MongoDB shell.

The result is that a DateTime is represented as either:

// in Strict output mode (i.e. Extended JSON):
{ "datetime" : { "$date" : 1522630923000 } } // 1522630923000 is the number of milliseconds since 1970-01-01T00:00:00
 
// in Shell output mode:
{ "datetime" : ISODate("2018-04-02T01:02:03Z") }

BsonDocument.Parse will read either of those correctly as a DateTime. But it will never convert a string to a DateTime.

You will either have to preprocess the input JSON strings to use the Extended JSON format used by MongoDB, or postprocess the resulting BsonDocument after parsing it (as you have also suggested in the previous comment). Preprocessing the strings is hard, post processing is relatively easy so that's the best solution.

I don't think we can change the behavior of the C# driver to look at every string it reads and see if it "looks" like a DateTime. That would be a backward breaking change and would be incompatible with what all other drivers and the MongoDB shell do.

Comment by Hugh Williams [ 02/Apr/18 ]

Workaround illustrated in an Xunit test (would still like to see BsonDocument.Parse do this for me though):

        [Fact]
        public void SerializeDate()
        {
            var now = DateTime.Parse("2018-04-01T23:43:24.1234567-04:00", CultureInfo.InvariantCulture, DateTimeStyles.AdjustToUniversal | DateTimeStyles.AssumeUniversal);
            var obj = new
            {
                now,
                inner = new
                {
                    now,
                    inmost = new
                    {
                        now,
                        array = new[]
                        {
                            now,
                            now,
                            now
                        },
                        array2 = new[]
                        {
                            new {now},
                            new {now}
                        }
                    }
                }
            };
            var json = Newtonsoft.Json.JsonConvert.SerializeObject(obj, Formatting.Indented);
            var bsonFromJson = BsonDocument.Parse(json);
            ConvertToIsoDates(bsonFromJson);
            var bson = obj.ToBsonDocument();
            _helper.WriteLine("JSON\n{0}", json);
            _helper.WriteLine("---");
            _helper.WriteLine("BSON from JSON\n{0}", bsonFromJson.ToJson(new JsonWriterSettings{Indent = true}));
            _helper.WriteLine("---");
            _helper.WriteLine("BSON\n{0}", bson.ToJson(new JsonWriterSettings { Indent = true }));
            Assert.Equal(bson.ToString(), bsonFromJson.ToString());
        }
 
 
        private static void ConvertToIsoDates(BsonDocument bsonFromJson)
        {
            for (var i = 0; i < bsonFromJson.ElementCount; ++i)
            {
                var bsonValue = bsonFromJson[i];
                switch (bsonValue.BsonType)
                {
                    case BsonType.String:
                        if (DateTime.TryParseExact(bsonValue.AsString, "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'FFFFFFFZ", CultureInfo.InvariantCulture, DateTimeStyles.None,
                                                   out var result))
                        {
                            bsonFromJson[i] = new BsonDateTime(result);
                        }
                        break;
                    case BsonType.Array:
                    {
                        ConvertToIsoDates(bsonFromJson[i].AsBsonArray);
                        break;
                    }
                    case BsonType.Document:
                        ConvertToIsoDates(bsonFromJson[i].AsBsonDocument);
                        break;
                }
            }
        }
 
 
        private static void ConvertToIsoDates(BsonArray bsonFromJson)
        {
            for (var i = 0; i < bsonFromJson.Count; ++i)
            {
                var bsonValue = bsonFromJson[i];
                switch (bsonValue.BsonType)
                {
                    case BsonType.String:
                        if (DateTime.TryParseExact(bsonValue.AsString, "yyyy'-'MM'-'dd'T'HH':'mm':'ss'.'FFFFFFFZ", CultureInfo.InvariantCulture, DateTimeStyles.None,
                                                   out var result))
                        {
                            bsonFromJson[i] = new BsonDateTime(result);
                        }
 
                        break;
                    case BsonType.Array:
                    {
                        ConvertToIsoDates(bsonFromJson[i].AsBsonArray);
                        break;
                    }
                    case BsonType.Document:
                        ConvertToIsoDates(bsonFromJson[i].AsBsonDocument);
                        break;
                }
            }
        }

Generated at Wed Feb 07 21:41:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.