[GODRIVER-1330] Document interface{} unmarshalling behavior Created: 03/Oct/19 Updated: 27/Oct/23 Resolved: 27/Mar/23 |
|
| Status: | Closed |
| Project: | Go Driver |
| Component/s: | BSON |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Quest Henkart | Assignee: | Unassigned |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | Documentation | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Documentation Changes Summary: | 1. What would you like to communicate to the user about this feature? |
||||||||
| Description |
|
EDIT: The resolution for this ticket is that we should document that unmarshalling into a struct that has a field with type interface{} yields a bson.D. This differs from mgo, which defaulted to bson.M.
if we have a struct like
In our case this struct is an array of objects. It was actually just a draftJS block. in JSON it would look like this, and this is what it would look like after being inserted into mongo
the mongo driver marshalled this perfectly into the database. No issues there. However when pulling the data out, we receive
this doesn't reflect the data in the database and actually makes no sense why mongo decided to break up the object like this |
| Comments |
| Comment by Esha Bhargava [ 27/Mar/23 ] | |
|
This has been documented here: "When unmarshalling, a field of type interface{} will follow the D/M type mappings listed above. BSON documents unmarshalled into an interface{} field will be unmarshalled as a D." You can configure to use bson.M : https://pkg.go.dev/go.mongodb.org/mongo-driver/bson#Decoder.DefaultDocumentM. | |
| Comment by Divjot Arora (Inactive) [ 14/Oct/19 ] | |
|
qhenkart@launchpadcentral.com I checked the mgo behavior and it defaults to unmarshalling as bson.M for an interface{} value. We are currently working on a documentation epic to improve our GoDoc docs and examples and this is definitely something we can document as part of that. I'll modify the description of this ticket to make sure this gets done. | |
| Comment by Quest Henkart [ 12/Oct/19 ] | |
|
@Divjot This is really something that you guys need to document as a breaking change from the community drivers to the official driver. It caused a severe bug in production because we had a []interface{} type in our struct, that upon changing to your driver, no longer represented the data that was in the database. I think the documentation should be clear that when Inserting a i := []interface{} and reading into an v := []interface{} i != v . There is nothing in the documentation that states reading into a []interface{} will break up each object in the array into key/value pairs and that the extendedJSON decoder must be used to read from the database but not to write into the database
imo. At least make it consistent. If you won't support []interfaces{} with standard json on read, then don't support it on write either. It's very misleading | |
| Comment by Divjot Arora (Inactive) [ 11/Oct/19 ] | |
|
qhenkart@launchpadcentral.com That behavior is expected. The driver is not converting the data, it is creating a bson.D instance that represents it. The stringified version of a bson.D should not be thought of as a one-to-one mapping of the actual document. Think of bson.D as an intermediate state. We chose not to implement a MarshalJSON function for bson.D because some BSON types cannot map directly to JSON types (e.g. JSON does not have an int32 vs int64 distinction). The extended JSON functions are meant to handle that issue as they follow the extended JSON specification, which essentially adds type information to ambiguous JSON types. | |
| Comment by Quest Henkart [ 11/Oct/19 ] | |
|
@Divjot I'm not totally understanding what you are saying. The entry in the example that is in mongo is standard JSON (not mongo's jsonext), It's just a simple untyped object. I don't understand why the mongo driver would convert that to Key/Value pair blocks that no longer represents the standard JSON that was entered. I would expect standard JSON entered into mongo to still be the same object (it is), and I would expect the same object in mongo to exit mongo as the same object and be marshaled into JSON as the same standard JSON (it's not)
I'm I misunderstanding or are you saying that https://gist.github.com/qhenkart/cb9f3e13c332ed8935e0eda8c5784f0c is expected behavior? | |
| Comment by Divjot Arora (Inactive) [ 11/Oct/19 ] | |
|
Hi qhenkart@launchpadcentral.com, Unmarshalling into interface{} creates a bson.D. The driver is not changing the format of the document, it is only creating a bson.D out of it. Inserting that bson.D into the database again would yield the same document as before. The string representation of a bson.D looks strange because we have not implemented a String function for it, so it makes it seem like there are extraneous keys, but those keys aren't actually part of the document.
| |
| Comment by Quest Henkart [ 11/Oct/19 ] | |
|
In my use case, I just changed our affected structs to use []map[string]interface{}. The reason I am making this report is because the functionality appears to be broken. Unmarshalling into a []interface{} should unmarshal any valid array from mongo as an untyped array. In this case, the driver is pulling out information from the database and incorrectly formatting it into its own primitive value representations that does not represent the values in the database nor does it represent the specification of a Go interface{}
For your suggestions, using bson.MarshalExtJSON does output the expected JSON, however there is still a major problem, because the issue was not the json.Marshal, it was the bson decoder. If you print out the contents of t here https://gist.github.com/qhenkart/cb9f3e13c332ed8935e0eda8c5784f0c#file-godriver-1330-example-L62, you will see that the value is now represented incorrectly (in the same way the example showed with json). And this makes processing valid information pulled from the database into an []interface{} impossible to process
| |
| Comment by Divjot Arora (Inactive) [ 09/Oct/19 ] | |
|
qhenkart@launchpadcentral.com The reason the output looks like this is because the Blocks field becomes a slice of bson.D. Because bson.D is defined as []bson.E and does not have a custom MarshalJSON function, using the json library to marshal it yields a JSON array of elements, each with a Key and Value. For your use case, I think you could use bson.UnmarshalExtJSON and bson.MarshalExtJSON instead of json.Unmarshal and json.Marshal, respectively. Can you try these functions and see if they yield better results? | |
| Comment by Quest Henkart [ 09/Oct/19 ] | |
|
@Divjot Arora . https://gist.github.com/qhenkart/cb9f3e13c332ed8935e0eda8c5784f0c | |
| Comment by Divjot Arora (Inactive) [ 03/Oct/19 ] | |
|
Hi qhenkart@launchpadcentral.com, can you send us a standalone code sample that reproduces this issue? |