[GODRIVER-988] Document decoding to empty interface Created: 21/Apr/19  Updated: 13/Jul/23

Status: Backlog
Project: Go Driver
Component/s: Documentation
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David Golden Assignee: Unassigned
Resolution: Unresolved Votes: 3
Labels: size-xsmall
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Quarter: FY24Q3
Case:

 Description   

A Reddit question about decoding SingleResult suggests that the driver will successfully decode into an empty interface variable.

func GetMonthStatusByID(ctx context.Context, id string) (interface{}, error) {
    var monthStatus interface{}
    filter := bson.M{"_id": id}
    err := db.Collection("Months").FindOne(ctx, filter).Decode(&monthStatus)
    return monthStatus, err
}

It looks like the result might have been a primative.D.

On one hand, this sort of makes sense because any type will satisfy the empty interface. OTOH, it's probably a user being misled by the type signature, as there's no real use for decoding to empty interface versus a concrete type or an Unmarshaler interface.

I think it should either error, on the grounds that users probably made a mistake, or else we should document the behavior to indicate that decoding to empty interface defaults to primative.D.



 Comments   
Comment by Srikant Varadarajan [ 25/Feb/20 ]

Hi Divjot

I have a related question about Decode
I have the a function say 
DoLookup()

{       var addr Address //Address is a struct    Lookup(filter, &addr) }

Lookup(filter bson.D, result interface{})

{    //do bunch of mongo related stuff.       s := collection.FindOne(ctx, filter)     err := s.Decode(&result)       logger.Printf("%+v", result)  //this outputs a key value map     logger.Printf("%T", result)   //this outputs primitive.D }

 
While calling Lookup, I pass a concrete struct (Address type). This is so that i can have a general purpose Lookup function but pass the concrete struct and return it back. 
 
This does not work.. since as i have indicated the result is a key value map.. and the Decode function does not decode it into the Address struct.
 
Not having this ability to pass the concrete type via an interface{} limits my ability to have a general purpose function collections. and instead having to write a identical code by replacing the interface with struct.
Is there something i am missing? is this an issue with the mongo go driver
 
PS: As an example.. when i use a similar pattern with json.Decode, it does the right thing..
 
 

Comment by Divjot Arora (Inactive) [ 09/Feb/20 ]

Hi jakirshaikh700@gmail.com,

Sorry for the confusion. We later realized that the advice given earlier in this ticket was not completely correct. We have addressed this issue in GODRIVER-1466, which was included in the recent 1.3.0 driver release. Would it be possible for you to upgrade to 1.3.0 and see if that fixes your issue?

 

– divjot

Comment by JAKIR SHAIKH [ 09/Feb/20 ]

Hello @Divjot Arora

I have tried a bellow solution but unfortunatly  it is not working for me.

tM := reflect.TypeOf(bson.M{})
reg := bson.NewRegistryBuilder().RegisterTypeMapEntry(bsontype.EmbeddedDocument, tM).Build()
clientOpts := options.Client().SetRegistry(reg)

My payload is having one attribute with interface type which is getting stored correctly but while decoding it is giving ambiguous result by placing all embedded attribute to the array ({ key: "xxx", value: "xxx"} )
could you please provide some pointers.

Comment by Raaz Crzy [ 02/Dec/19 ]

Another approach to solve this can be by first decoding it into bson.M type and then unmarshalling it to your struct. Yes, this is not optimal.

eg:

func GetMonthStatusByID(ctx context.Context, id string) (interface{}, error) {
 var monthStatus interface{}
 filter := bson.M\{"_id": id}
 err := db.Collection("Months").FindOne(ctx, filter).Decode(&monthStatus)
 return monthStatus, err
}

The above snippet should be changed to:

func GetMonthStatusByID(ctx context.Context, id string) (interface{}, error) {
 var monthStatus interface{}
 filter := bson.M\{"_id": id}
tempResult := bson.M{}
 err := db.Collection("Months").FindOne(ctx, filter).Decode(&tempResult)
if err == nil {   
    obj, _ := json.Marshal(tempResult)
    err= json.Unmarshal(obj, &monthStatus)
}
 return monthStatus, err
}

Comment by Pawan Sharma [ 06/Nov/19 ]

@Divjot Arora : It worked for me, thanks a lot!

Comment by Divjot Arora (Inactive) [ 05/Nov/19 ]

Hi pawan.sharma2@go-mmt.com,

In general, we recommend using concrete types if you know what your data looks like as this will preserve type safety. If you need to unmarshal data as interface{} and have it look like JSON when it's printed out, I think the following should work:

tM := reflect.TypeOf(bson.M{})
reg := bson.NewRegistryBuilder().RegisterTypeMapEntry(bsontype.EmbeddedDocument, tM).Build()
clientOpts := options.Client().SetRegistry(reg)

when creating a new Client. The reason for this is that the driver defaults to unmarshalling as bson.D for interface{} where as mgo defaults to bson.M. The above code should match mgo behavior.

Comment by Pawan Sharma [ 05/Nov/19 ]

Hi, Any updates here?

Comment by Thomas Coussot [ 13/May/19 ]

Hi!

 

I have a similar problem. I need to store a complex JSON object (actually a JSON array) and so I defined a struct which look like this:

 

type Survey struct {
    ID       primitive.ObjectID `json:"id" bson:"_id,omitempty"`
    Name     string             `json:"name"`
    FormData interface{}        `json:"formData" bson:"formData"`
}

(I also tried with []interface{})

 

 

My values look like this (raw JSON before json.Decode, and also what ends up in the DB):

{
    "id": "5cd4288cdb54ec6e4b14550f",
    "name": "A name",
    "formData": [
        {
            "type": "title",
            "text": "My title"
        },
        {
            "type": "text",
            "text": "quick intro to the survey"
        },
        {...},
    ]
}

An important factor is that my elements don't always have a "text" field and can have other fields (children, options, label, ...). Also, elements can be nested so unstructured JSON really suits my needs.

 

As indicated, storing data with this struct goes smoothly.

However, when retrieving data, I end up with the same kind of problem as this Issue:

 

{
    "id": "5cd4288cdb54ec6e4b14550f",
    "name": "A name",
    "formData": [
      [
        {
          "Key": "text",
          "Value": "My title"
        },
        {
          "Key": "type",
          "Value": "title"
        }
      ],
      [
        { "Key": "text", "Value": "quick intro to the survey" },
        { "Key": "type", "Value": "text" }
      ],
      [...],
    ]
}

 

The driver seems to not properly decode the document into the `interface{}` as it doesn't know which type it is.

The unofficial Mongo driver (mgo) seems to handle this fine (we have used it for this several times before)

 

It's kinda deceiving not being able to retrieve unstructured JSON with an empty interface, just as json.Decode and mgo let us.

 

The only solution I found ATM is to use a different type for data, which results in a byte array in the DB.

 

FormData         json.RawMessage    `json:"formData" bson:"formData"`

Obviously, by doing so, data aren't readable when directly looking in the DB, which breaks interoperability and is just ugly.

 

I see you said that "there's no real use for decoding to empty interface", is there another way to do what I want?

How can I store and retrieve a plain JSON object in a struct member?

 

Best regards

 

Generated at Thu Feb 08 08:35:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.