[JAVA-280] BSONTimestamp and some other fields are not properly serialized to JSON and cannot be parsed back Created: 23/Feb/11  Updated: 19/Oct/16  Resolved: 25/Apr/11

Status: Closed
Project: Java Driver
Component/s: None
Affects Version/s: 2.4
Fix Version/s: 2.5.3

Type: Bug Priority: Minor - P4
Reporter: Ryan Nitz Assignee: Antoine Girbal
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-2684 special types in shell (e.g. Timestam... Closed

 Description   

Calling toString() on a BSON object that has a BSONTimestamp field, yields a display of:

"start" : 1298475384|324



 Comments   
Comment by auto [ 29/Mar/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: JAVA-280: bug in BSONTimestamp JSON parse. Added BSON encoding of DBRef. Added test for JSON encode/decode all types.
https://github.com/mongodb/mongo-java-driver/commit/a15a995d9a1485cbf059345d6863af53b1542f7f

Comment by auto [ 22/Mar/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: JAVA-280: NaN double value is not properly parsed back in JSON
https://github.com/mongodb/mongo-java-driver/commit/17cf46df360896ad332ca63c9a2bd7698327f807

Comment by auto [ 19/Mar/11 ]

Author:

{u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}

Message: JAVA-280: BSONTimestamp and some other fields are not properly serialized to JSON and cannot be parsed back
https://github.com/mongodb/mongo-java-driver/commit/5263bbeda933d50185745a99b66479e4539ffc99

Comment by Antoine Girbal [ 18/Mar/11 ]

After the fix, the following document is properly parsed from json, inserted in db, read from db, serialized into json:

{
"str" : "asdfasd" ,
"long" : 5 ,
"dble" : 0.4 ,
"bool" : false ,
"date" :

{ "$date" : "2011-05-18T18:56:00Z"}

,
"pat" :

{ "$regex" : ".*" , "$options" : ""}

,
"oid" :

{ "$oid" : "4d83ab3ea39562db9c1ae2ae"}

,
"ref" : { "$ref" : "test.test" , "$id" : { "$oid" : "4d83ab59a39562db9c1ae2af"}} ,
"code" :

{ "$code" : "asdfdsa"}

,
"codews" : { "$code" : "ggggg" , "$scope" : { }} ,
"ts" :

{ "$ts" : 1300474885 , "$inc" : 3}

,
"null" : null
}

Notes:

  • json encoding/decoding was fixed for DBRef, Code, BSONTimestamp
  • the timestamp representation still shows a number, cause it's faster to encode / decode. By comparison the date field has a lot of overhead in java (creates a date formatter for every date to be written). If you want a pretty print you can always call toString() on the timestamp object.
  • the json parser was storing numbers with less than 10 digit in an int, otherwise in a long. I think this is arbitrary and can lead to bugs later on. When reading a stream of objects for a collection, they should have same type for a given field, so it should always be long.
  • the Types.getByte() was totally not complete compared to BSONEncoder. Fixed to match.

Let me know if sounds ok and I'll commit.

Comment by Antoine Girbal [ 17/Mar/11 ]

Here are most types we support:
"str" : "" ,
"int" : 0 ,
"long" : 0 ,
"float" : 0.0 ,
"dble" : 0.0 ,
"bool" : true ,
"date" :

{ "$date" : "2011-03-17T21:42:42Z"}

,
"pattern" :

{ "$regex" : "" , "$options" : ""}

,
"oid" :

{ "$oid" : "4d828076a395ceb40b028102"}

,
"bin" : <Binary Data> , -> not parseable, that's ok
"codeWScope" :

{ "$code" : "" , "$scope" : null }

,
"code" : "", -> parseable but becomes string
"ts" : 1300398223|0 -> not parseable

I guess I'll fix so that whatever is serialized can be parsed, besides Binary.

Comment by Antoine Girbal [ 17/Mar/11 ]

Also, it's kind of bad that you cannot do mongoexport / mongoimport and get things identical

foo:PRIMARY> db.accounts.insert(

{a: Timestamp()}

)
foo:PRIMARY> db.accounts.find()
{ "_id" : ObjectId("4d825eeef7e844daacb16e96"), "a" :

{ "t" : 1300389614000, "i" : 1 }

}
here I'm assuming the timestamp has type "timestamp" in the bson.

$ ./mongoexport -d testdb -c accounts | ./mongoimport -d testdb -c accounts2

foo:PRIMARY> db.accounts2.find()
{ "_id" : ObjectId("4d825eeef7e844daacb16e96"), "a" :

{ "t" : NumberLong("1300389614000"), "i" : 1 }

}
here I'm assuming the timestamp is just an object in the bson..

For all native bson types, we should make sure they can be exported/imported in JSON reliably.

Comment by Antoine Girbal [ 17/Mar/11 ]

As other example, regexp is like:

{ "$regex" : ".*" , "$options" : ""}
Comment by Antoine Girbal [ 17/Mar/11 ]

so there are 2 ways the timestamp is represented today:

  • with toString(): TS time:Thu Mar 17 11:52:18 PDT 2011 inc:0 -> nice enough
  • with JSON.serialize(): 1300387938|0 -> not good

when doing dbobject.toString(), it will call the JSON.serialize.
The serialized representation doesnt really need to be pretty, but right now the bigger issue is that the JSON cannot be parsed back.
So we should change that to smthing parseable.
A date is represented in a way that's easy to parse back:

{ "$date" : "2011-03-17T19:00:09Z"}

If you look in shell, a timestamp is represented by an object with no easy way to tell it's a timestamp:
foo:PRIMARY> a =

{field: Timestamp()}

{ "field" :

{ "t" : 0, "i" : 0 }

}

So I think the format should be either:

  • like shell: { "t" : 0, "i" : 0 }

    , but then it's not clean to parse back, and not consistent with rest of driver

  • like java date: { "$ts" : "2011-03-17T19:00:09Z", "$inc:": 123}
Comment by Ryan Nitz [ 17/Mar/11 ]

Nod... they recommend using either commas or decimal points for fractions.

http://en.wikipedia.org/wiki/ISO_8601#Times

Comment by Scott Hernandez (Inactive) [ 17/Mar/11 ]

That won't display the increment part though, will it? Sometimes it is very useful to see that when looking at a bunch from the same second.

What about Timestamp("<iso string>", ###)? We could make this universal with javascript and other drivers.

Comment by Ryan Nitz [ 16/Mar/11 ]

How about using the ISO 8601 format?

http://en.wikipedia.org/wiki/ISO_8601

Comment by Eliot Horowitz (Inactive) [ 26/Feb/11 ]

Any suggestions on the correct display?

Generated at Thu Feb 08 08:51:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.