[GODRIVER-1967] Pipeline can't handle empty structs and maps Created: 16/Apr/21  Updated: 27/Oct/23  Resolved: 06/May/21

Status: Closed
Project: Go Driver
Component/s: API
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Unknown
Reporter: Sean Teeling Assignee: Benji Rewis (Inactive)
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by GODRIVER-1968 Pipeline can't handle empty structs a... Closed
Related
is related to SERVER-54046 Permit empty objects as expressions i... Closed

 Description   

collection.UpdateOne fails if a {{$set, myStruct}} is supplied within a mongo pipeline, and the struct contains empty values.

 

However that same $set succeeds, when it's not within a pipeline.



 Comments   
Comment by Backlog - Core Eng Program Management Team [ 06/May/21 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by Benji Rewis (Inactive) [ 21/Apr/21 ]

We really do apologize for any confusion and sincerely hope you continue to use MongoDB teeling.sean@gmail.com!

After doing some more digging, we’ve found there is a ticket (SERVER-54046) already investigating whether permitting empty objects as expressions in $set or $addFields might be a worthwhile improvement to the core server. I asked for an update on 54046, since it’s currently backlogged.

It is indeed up to the server whether to include that improvement, so there’s not much to be done on the driver end.

Comment by Sean Teeling [ 20/Apr/21 ]

Lastly, go recommends using a map to the empty struct to replicate set behavior, which is incompatible here. I don't care if it's the diver or not. Let's get a fix in (even if the driver has to make those edits on the users behalf, or provides an option to do so). 

 

I can't imagine ever providing an API like this to my users.. they would shoot me dead on the spot

Comment by Sean Teeling [ 20/Apr/21 ]

Can you imagine my frustration debugging something like this, investing time into mongoDB only to find there are more rough edges than smooth edges. Then I bring up a real issue (please don't tell me this is working as intended, this is insane). and instead of offering a fix, or opening a tracking bug, I'm told I need to come up with my own work around to convert to a bson.D doc.

 

Let's create beautiful, user-friendly API's.. not whatever this is. I'm about ready to rip out any mongo code and go with another provider at this point, and never look back at this hot mess

 

Comment by Benji Rewis (Inactive) [ 20/Apr/21 ]

Hello teeling.sean@gmail.com! Thanks again for your report. I think the error you’re getting is a server error (not from the Go driver) and is expected behavior.

In the Go driver, an empty map like the one you're setting in Field5 (map[string]string{}) will marshal into BSON as {"$set": {“Field5”: {}}}.

$set has two versions. The first is the field update operator (docs here). This is the $set used with UpdateOne and a regular bson.D (not a mongo Pipeline) as the update argument. This $set allows fields with empty objects such as {“Field5”: {}}.

The second version of $set is the aggregation pipeline stage (docs here). This is the $set used with UpdateOne and a mongo.Pipeline as the update argument. This $set is based off $project, which as of MDB version 3.4, does not support fields with empty objects such as {“Field5”: {}}. You’ll notice that setting Field5 to nil will work, as that marshals into BSON as {"$set": {“Field5”: null}}, so there are no fields with empty objects within the $set aggregation pipeline stage.

If you need an empty object value within a $set, you’ll have to Update with a bson.D and not a mongo.Pipeline. Let me know if that doesn't make sense/you have any questions!

Comment by Sean Teeling [ 20/Apr/21 ]

I also tried with a map[string]struct{}, with a single value (which has an empty struct), which had issues as well

Comment by Sean Teeling [ 19/Apr/21 ]

package mainimport (
	"context"
	"fmt"	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
	"go.mongodb.org/mongo-driver/mongo/options"
)var (
	collection *mongo.Collection
)type ChangeEvent struct {
	FullDocument map[string]interface{}
}type Temp struct {
	Field1 string
	Field2 string
	Field3 int
	Field4 map[string]int
	Field5 map[string]string
}func Create() {
	t := Temp{
		Field1: "val1",
		Field3: 2,
		Field5: map[string]string{},
	}
	res, err := collection.InsertOne(context.Background(), t)
	if err != nil {
		panic(err)
	}
	fmt.Println(res.InsertedID)
}func Update() {
	filter := bson.M{"field1": "val1"}
	t := Temp{
		Field1: "new hello!",
		Field2: "This shouldn't show up...",
		Field3: 4,
		Field5: map[string]string{},
	}
	_, err := collection.UpdateOne(context.Background(), filter,
		mongo.Pipeline{bson.D{{Key: "$set", Value: t}}}) // this works if we supply a bson.D, instead of a mongo.Pipeline
	if err != nil {
		panic(err)
	}}func main() {
	opts := options.Client().ApplyURI("mongodb://mongodb:27017")
	client, err := mongo.Connect(context.Background(), opts)
	if err != nil {
		panic(err)
	}	collection = client.Database("temp").Collection("temp")
	Create()
	Update()
	fmt.Println("success!")
}

 

Paste-formatting didn't turn out great, but this reproduces with the error:

 

multiple write errors: [{write errors: [

{Invalid $set :: caused by :: an empty object is not a valid value. Found empty object at path field5}

]}, {<nil>}]

 

In particular, leaving the map[string]string, as nil works, but creating an empty map causes the error. Removing the update document from the mongo pipeline also makes the above code work.

Comment by Kevin Albertson [ 19/Apr/21 ]

Hi teeling.sean@gmail.com, thank you for the report! We will look into this soon.

Do you have a reproducible example handy? It may help us investigate to know how myStruct is getting marshalled into BSON, as well as what the exact error is (whether it is a server-side error or not).

As a quick test, I attempted an UpdateOne operation with both a bson.D and mongo.Pipeline as the update argument using an empty struct as the value, and both updated as expected.

package investigations
 
import (
	"context"
	"fmt"
	"testing"
 
	"go.mongodb.org/mongo-driver/bson"
	"go.mongodb.org/mongo-driver/mongo"
)
 
type MyTestStruct struct {
	A int32
}
 
func Test1967(t *testing.T) {
	client, err := mongo.Connect(context.TODO())
	if err != nil {
		t.Error(err)
	}
	coll := client.Database("db").Collection("coll")
	err = coll.Drop(context.TODO())
	if err != nil {
		t.Error(err)
	}
	_, err = coll.InsertOne(context.TODO(), bson.D{{"_id", 0}})
	if err != nil {
		t.Error(err)
	}
	// Either of these works, setting the value of "a" to 0:
	res, err := coll.UpdateOne(context.TODO(), bson.D{{"_id", 0}}, bson.D{{"$set", MyTestStruct{}}})
	// res, err := coll.UpdateOne(context.TODO(), bson.D{{"_id", 0}}, mongo.Pipeline{bson.D{{"$set", MyTestStruct{}}}})
	if err != nil {
		t.Error(err)
	}
	fmt.Printf("Matched %v and modified %v\n", res.MatchedCount, res.ModifiedCount)
}

Generated at Thu Feb 08 08:37:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.