-
Type: Bug
-
Resolution: Unresolved
-
Priority: Unknown
-
Affects Version/s: None
-
Component/s: None
-
None
-
Go Drivers
Detailed steps to reproduce the problem?
This occurs when the batch logic partitions data p1,p2,...,pn and fails when advancing pj where j > 1. AFAICT this only happens when one of the batches contains a document that is too large. That is, the error is not a WriteCommandError, checked here. You can use the following to reproduce this behavior:
package main import ( "context" "fmt" "log" "go.mongodb.org/mongo-driver/v2/bson" "go.mongodb.org/mongo-driver/v2/mongo" ) func main() { client, err := mongo.Connect() if err != nil { panic(err) } defer func() { client.Disconnect(context.Background()) }() coll := client.Database("db").Collection("coll") docs := make([]any, 100) for i := range docs { docs[i] = bson.D{{"x", 1}} } docs = append(docs, bson.D{{"x", string(make([]byte, 16_777_150))}}) fmt.Println("len before adding breaker: ", len(docs)) docs = append(docs, bson.D{{"y", string(make([]byte, 17_000_000))}}) _, err = coll.InsertMany(context.Background(), docs) if err != nil { log.Println("err: ", err) } cur, err := coll.Find(context.Background(), bson.D{}) if err != nil { log.Fatalf("failed to execute find command: %v", err) } var found []bson.D if err := cur.All(context.Background(), &found); err != nil { log.Fatalf("failed to decode data: %v", err) } fmt.Println("found: ", len(found)) }
Definition of done: what must be done to consider the task complete?
There are two obvious solutions:
- We could check all document sizes when advancing the initial batch and return an error if any of the documents exceed the max length. This would ensure that nothing is inserted. This might be considered a breaking change.
- We could create a new error type that contains an WriteError that can be returned to the mongo layer. We can then include the results but remove the ids for the errors defined by the index of the write error, i.e. large documents.
The exact Go version used, with patch level:
go version go1.23.1 darwin/arm64
The exact version of the Go driver used:
1.17.1
Describe how MongoDB is set up. Local vs Hosted, version, topology, load balanced, etc.
Using MongoDB: 7.0.9 Using Mongosh: 2.3.1
The operating system and version (e.g. Windows 7, OSX 10.8, ...)
OSX 14.5
Security Vulnerabilities
NA
- is related to
-
GODRIVER-2388 Improved Bulk Write API
- In Progress
-
DRIVERS-3025 Add guidance on batching data for insert/update/delete commands
- Needs Triage
- related to
-
GODRIVER-982 What errors should be returned when a document in a batch is too large?
- Closed