[CSHARP-1618] BsonDocument.Parse fails for some valid ISODate values Created: 04/Apr/16  Updated: 23/Sep/16  Resolved: 13/Apr/16

Status: Closed
Project: C# Driver
Component/s: BSON
Affects Version/s: 1.11, 2.0.1
Fix Version/s: 2.3

Type: Bug Priority: Minor - P4
Reporter: Yaniv Weinberg Assignee: Robert Stam
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 2012



 Description   

VMware is using Mongo as part of its' Mirage product.
We are parsing the rs.status() state to keep track of the Mongo health and display this information in our UI.
As part of the parsing we use BsonDocument.Parse method to parse the relevant rs.status() block.
This was failing at a customer system:

---> System.FormatException: String was not recognized as a valid DateTime format
in System.DateTime.ParseExact(String s, String[] formats, IFormatProvider provider, DateTimeStyles style)
in MongoDB.Bson.IO.JsonReader.ParseISODateTimeConstructor()
in MongoDB.Bson.IO.JsonReader.ReadBsonType[TValue](BsonTrie`1 bsonTrie, Boolean& found, TValue& value)
in MongoDB.Bson.Serialization.Serializers.BsonDocumentSerializer.Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
in MongoDB.Bson.Serialization.Serializers.BsonBaseSerializer.Deserialize(BsonReader bsonReader, Type nominalType, IBsonSerializationOptions options)
in MongoDB.Bson.BsonDocument.Parse(String json)
in Wanova.Server.Common.MongoServices.MongoShellOperations.ExtractDataFromShell(String result)
— Fine della traccia dello stack dell'eccezione interna —
in Wanova.Server.Common.MongoServices.MongoShellOperations.ExtractDataFromShell(String result)
in Wanova.Server.Common.MongoServices.MongoConfigurer.UpdateReplSetStatus(IList`1 allNode)
in Wanova.Server.Common.Management.Server.Services.MongoServiceRecoveryTask.UpdateServiceState()

This caused our system to not recognize the Mongo state correctly.
After some research we found out this happend because the settings for the time format were: HH.mm.ss
This caused the exception and parsing to fail.
We haven't done much additional research as the case is resolved, this is just "FYI"



 Comments   
Comment by Githook User [ 13/Apr/16 ]

Author:

{u'name': u'Robert Stam', u'email': u'Robert Stam'}

Message: CSHARP-1618: Fixed minor bug.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/af0707c66cd0df113e3e6926e271a618994c2b26

Comment by Githook User [ 13/Apr/16 ]

Author:

{u'name': u'Robert Stam', u'email': u'Robert Stam'}

Message: CSHARP-1618: Parse additional valid ISODate values.
Branch: master
https://github.com/mongodb/mongo-csharp-driver/commit/b13dffb121e7b3d10f9cb7802c78296f59e04523

Comment by Robert Stam [ 04/Apr/16 ]

The initial assumption was that this exception was being thrown because the time component was using "." instead of ":" as a separator.

According to ISO 8601 "." is not a valid separator. It can only be used to indicate fractions (usually fractions of a second).

Another explanation might be that the string had only "HH:mm" (i.e no seconds or fractions of a second).

We don't parse that correctly, even though it is valid according to ISO 8601 (the seconds are implied to be zero).

There are other ISO 8601 variants we don't parse correctly either (for example, separators are optional).

At the very least we should parse any variant we produce ourselves (either by a driver or any command line tool like the shell or mongoexport).

It might be worth while supporting additional valid ISO 8601 formats, even the ones we might never produce ourselves.

Comment by Yaniv Weinberg [ 04/Apr/16 ]

rs.status() returns the above block when run manually or through the code, this seems like a normal format.
The function receives the above document exactly, with the : in the format.
The issue occurs inside the parsing code with the stacktrace indicated above.
Maybe it calls the system time at some point and gets a HH.mm format?

Anyway this is an FYI case, the issue is resolved once they moved to HH:mm

Comment by Robert Stam [ 04/Apr/16 ]

The Shell syntax:

ISODate("date-value")

requires that the "date-value" string be in ISO 8601 format, which requires ":" as the separator (any local region settings on the computer are supposed to be ignored).

Not sure how a document with an invalid "date-value" came into existence, but if it is not using ":" as the separator in "hh:mm:ss" then it is not a valid "date-value".

Comment by Yaniv Weinberg [ 04/Apr/16 ]

I don't get it either, the customer has before we changed their format to HH:mm:ss and HH:mm
Yes, in was an Italian system.
But specifically with the HH.mm.ss time format (As just adding english language didn't fix the problem).
Maybe it is a combination of Italian system and the HH.mm.ss format.

Comment by Robert Stam [ 04/Apr/16 ]

I don't get any exception when parsing that string with BsonDocument.Parse.

Do you mean that you get an exception parsing that string on a system with the regional settings set to Italian?

Comment by Yaniv Weinberg [ 04/Apr/16 ]

I just guess that there are other formats which may break this other than "." as separator.
You never know what customers are using.
Regarding the parsing, you are right, I will bring this up with our dev team, but Bson.Document should work on all systems as the document is of a good format.

An example of what it couldn't parse on a system with the "." seperator.
Also note this was a system in Italian (All menus and everything).

{
	"set" : "mirageRS",
	"date" : ISODate("2016-04-04T11:51:29.979Z"),
	"myState" : 1,
	"members" : [
		{
			"_id" : 0,
			"name" : "SRVMIRAGE01:27017",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 520,
			"optime" : Timestamp(1459769209, 1),
			"optimeDate" : ISODate("2016-04-04T11:26:49Z"),
			"electionTime" : Timestamp(1459770170, 1),
			"electionDate" : ISODate("2016-04-04T11:42:50Z"),
			"configVersion" : 1,
			"self" : true
		}
	],
	"ok" : 1
}

The editor has wonked up the spaces a bit, sorry

Comment by Robert Stam [ 04/Apr/16 ]

I'm not sure about relying on the regional settings... it doesn't necessarily agree with the representation used by the data.

Normally all data stored in or coming from a database should be in a culture invariant format. It should be up to the presentation layer to convert to local region and timezone for display purposes.

In this case it looks like you are invoking the shell to execute "rs.status()" and then capturing and parsing what was sent to standard output. Another approach would be to directly execute whatever command(s) rs.status() calls, which would avoid having to parse a string at all.

Regarding the parsing error, a good general rule is to be strict when writing data and forgiving when reading it, so it seems like we could relax our parsing code to work with either ":"or "." as the separator (regardless of region settings).

Comment by Craig Wilson [ 04/Apr/16 ]

Could you provide an example document that failed to parse so we can reproduce this?

Comment by Yaniv Weinberg [ 04/Apr/16 ]

I mean the HH.mm.ss format instead of HH:mm:ss
This could also not accept other formats as well.
I think we should just query the windows regional settings for correct parsing on all systems
Thanks!

Generated at Wed Feb 07 21:40:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.