[GODRIVER-1596] Issues with Time unmarshalling Created: 29/Apr/20  Updated: 27/Oct/23  Resolved: 30/Apr/20

Status: Closed
Project: Go Driver
Component/s: BSON, JSON & ExtJSON
Affects Version/s: 1.2.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Abhay Kumar Assignee: Divjot Arora (Inactive)
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

go version go1.13.3 windows/amd64


Attachments: PNG File image002.png    
Issue Links:
Duplicate
is duplicated by GODRIVER-1597 Issues with Time unmarshalling Closed

 Description   

Version : go version go1.13.3 windows/amd64
Mongo Driver Version : go.mongodb.org/mongo-driver v1.2.1

I'm fetching the Date stored in MongoDB. The value stored in DB is
reservedDate: 2020-02-19T07:11:23.890+00:00

After Unmarshalling the response into time.Time field the value becomes
"reservedDate": "2020-02-19T07:11:23.89Z"

890 MilliSecond become 89, similarlly 100 becomes 1. The trailing Zeros are getting truncated.



 Comments   
Comment by Abhay Kumar [ 01/May/20 ]

Thank you Divjot. It clarifies things. I will watch the golang issue, hope they add this support soon. 

Regards

Abhay Kumar

 

Comment by Divjot Arora (Inactive) [ 01/May/20 ]

abhay.kumar1@pb.com I don't think this is possible. Going from the value stored in the database to the JSON response you send out of your application requires two steps:

  1. Fetching the value from the database, which is a BSON datetime, and unmarshalling it into a Go time.Time value.
  2. Marshalling the struct containing the time.Time into JSON.

The driver is only responsible for the first step. After you have the time.Time in your struct, we cannot control how it stringifies itself. That is controlled by Go's encoding/json library. Also, the time.Time type has a custom MarshalJSON function, which always marshalls the value as an RFC3339 string with the minimal fraction for seconds (i.e. .89 rather than .890). I'm not aware of a way for any user to control the format of the JSON time string.

If you're interested, there is a proposal to add struct tags to modify this behavior for the encoding/json library when marshalling/unmarshalling time.Time values: https://github.com/golang/go/issues/21990.

Hopefully this clarifies things. Let me know if you have any other questions about this.

– Divjot

Comment by Abhay Kumar [ 01/May/20 ]

I know you have closed it, can there be a way to specify the pattern/format for decoding time value.

When we do cur.Decode(&activity)

 

May be something like bson:"activityDate" pattern:"2006-01-02T15:04:05.000Z"

type activity Struct

{ ActivityDate         time.Time           `json:"activityDate" bson:"activityDate"` }

 

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

Thanks for the links abhay.kumar1@pb.com. I'm closing out this issue as "Works as Designed". Feel free to leave another comment or open a new issue if you have any other questions.

Comment by Abhay Kumar [ 30/Apr/20 ]

https://github.com/golang/go/issues/38778

Sent from Outlook Mobile<https://aka.ms/blhgte>

Comment by Abhay Kumar [ 30/Apr/20 ]

https://github.com/golang/go/issues/37293

Sent from Outlook Mobile<https://aka.ms/blhgte>

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

Out of curiosity, can you provide links to the previous and new Go issues? I'd like to follow them as well.

Comment by Abhay Kumar [ 30/Apr/20 ]

Thank you, I had earlier logged the issue with GO. They closed the issue saying they thought it was Mongo Driver error. But your example explains that it the behavior of GO. I've raised the issue with GO again, 

 

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

I understand that this is an inconvenience, but given that we can replicate the issue without using the driver at all, it seems like the underlying issue is that Go prints out the minimal fraction needed to correctly represent the time and requires a call to time.Time.Format to get the exact format you want. I don't think there's anything the driver can do about this, though, because we only create the time.Time instance and cannot control how it stringifies itself when marshalled as JSON. If you agree with this, I can close out this ticket.

– Divjot

Comment by Abhay Kumar [ 30/Apr/20 ]

I use the struct and decode the value directly from bson to struct

type activity struct

{ ActivityDate         time.Time    `json:"activityDate" bson:"activityDate"` }

 
and write this directly back in the writer 
json.NewEncoder(w).Encode(activity)
 
Now I will have to format the date value first and then change the value in the response. 
 
Since the millisecond didn't really matter too my code I am truncating the millisecond values
time.Now().UTC().Truncate(time.Second)
 
go does not give an error when you do
layout := "2006-01-02T15:04:05.000Z"layout := "2006-01-02T15:04:05.000Z" date, err := time.Parse(layout, timeString)
 
however other programming languages like kotlin date formatter gives exception
 

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

The fact that time.Time.String() prints the time as .89 instead of .890 isn't something the driver can control. If you need to send the time back in a JSON response with a certain format, can you use time.Time.Format? I believe this should work: https://play.golang.org/p/7aiBIOaRwbz.

Comment by Abhay Kumar [ 30/Apr/20 ]

The behavior causes a problem when you send this value back as JSON string. When you parse you have to specify the format.

For example, in go I define my 

 
layout := "2006-01-02T15:04:05.000Z"
and I parse the date 

 
date, err := time.Parse(layout, dateStr)
if the dateStr is 2020-02-19T07:11:23.890+00:00 it will parse succefully else it will give error if the dateStr is 2020-02-19T07:11:23.89+00:00 

Even bigger when 2020-02-19T07:11:23.800+00:00 will beocme 2020-02-19T07:11:23.8+00:00

 

 

 

 

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

I think I know what's happening here. I think the .890 is the fractional part of a second, meaning 890/1000 of a second, or 890 milliseconds. However, trailing zeroes after a decimal point are not necessary and this is the same as 89/100 of a second, so Go prints out the time as .89

This code example should show that Go's time.Parse will preserve all three digits only when it's necessary: https://play.golang.org/p/fjjssOCXpsr.

Comment by Abhay Kumar [ 30/Apr/20 ]

Thanks for writing the code. I see you have replicated the issue successfully

You have got the mismatch in your response in the code you have shared. The const time you have declared is

timeString = "2020-02-19T07:11:23.890+00:00"

The response you have got is

ActivityDate: (time.Time) 2020-02-19 07:11:23.89 +0000 UTC

890 became 89

The zero was truncated its should have been 890 milliseconds it became 89 milliseconds

 

Comment by Abhay Kumar [ 30/Apr/20 ]

You have got the mismatch in your response in the code you have shared. The const time you have declared is

timeString = "2020-02-19T07:11:23.890+00:00"

The response you have got is

ActivityDate: (time.Time) 2020-02-19 07:11:23.89 +0000 UTC

890 became 89

The zero was truncated its should have been 890 milliseconds it became 89 milliseconds

 

 

Comment by Divjot Arora (Inactive) [ 30/Apr/20 ]

Hi abhay.kumar1@pb.com,

Apologies if this wasn't clear in my last comment, but we'd like a minimal code sample that we can run to reproduce this issue, similar to the guidelines outlined in https://stackoverflow.com/help/minimal-reproducible-example. This is clearer than a list of steps, which can be ambiguous. Specifically, I had two issues reproducing this with the steps you outlined:

  1. The value "2020-02-19T07:11:23.890+00:00" and adding it as a time is unclear. Was this added using Go code via time.Parse or by some other tool (e.g. another language, shell, Compass, etc)
  2. In the lsat step, the value gets stringified differently depending on how the logging is done. When I use fmt.Println to print out a time.Time value, it prints in the format "2020-02-19 07:11:23.89 +0000 UTC", which seems to be different than the results you're getting.

This was my unsuccessful attempt at writing code for your steps: https://play.golang.org/p/KOMlQY8m_o_K.

– Divjot

Comment by Abhay Kumar [ 30/Apr/20 ]
  • Create a collection to have a time field
  • Add the value 2020-02-19T07:11:23.890+00:00
  • Create a struct to hold the value
  • type Activity struct {  ActivityDate         time.Time           `json:"activityDate" bson:"activityDate"` }

    craete a var act Activity

  • do the Collection.FindOne().Decode(&act)
  •  Log the value of the ActivityDate that comes back
Comment by Divjot Arora (Inactive) [ 29/Apr/20 ]

abhay.kumar1@pb.com If possible, it would also be helpful if you can send us a minimal example that we can use to reproduce this.

Comment by Divjot Arora (Inactive) [ 29/Apr/20 ]

Hi abhay.kumar1@pb.com,

Can you provide the following information to help us debug:

  1. How are you printing the value stored in the DB? Are you using the shell, extended JSON, or some other tool?
  2. You mention that you're unmarshalling the value into a time.Time field and that prints as "2020-02-19T07:11:23.89Z". However, when I try to print a time.Time value using fmt.Println, it prints as "2020-04-29 20:27:27.24 +0000 UTC". Can you explain how you're printing the Go value to get that format?
Generated at Thu Feb 08 08:36:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.