[GODRIVER-1363] InsertMany returns InsertedID's from insertions that errored Created: 19/Oct/19  Updated: 28/Oct/23  Resolved: 16/Apr/20

Status: Closed
Project: Go Driver
Component/s: CRUD
Affects Version/s: 1.1.2
Fix Version/s: 1.3.3

Type: Bug Priority: Major - P3
Reporter: Joël Gähwiler Assignee: Isabella Siu (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

When inserting a document using Collection.InsertMany that fails due to a unique index constraint the resulting mongo.InsertManyResults reports the documents ID as being Inserted.

It looks like that the results are actually computed before issuing the BulkWrite operation and never updated according to its outcome. Cleary, the results of the BulkWrite should be taken into account and only the IDs of successfully inserted documents may be returned together with the last error.

Here the result from the mentioned call:

&mongo.InsertManyResult{
    InsertedIDs: {
        primitive.ObjectID{0x5d, 0xab, 0x36, 0x86, 0xf, 0xa5, 0xbc, 0x20, 0x30, 0xd6, 0x1c, 0x8},
    },
} mongo.BulkWriteException{
    WriteConcernError: (*mongo.WriteConcernError)(nil),
    WriteErrors:       {
        {
            WriteError: mongo.WriteError{Index:0, Code:11000, Message:"E11000 duplicate key error collection: test.test index: _id_ dup key: { : ObjectId('5dab36860fa5bc2030d61c08') }"},
            Request:    nil,
        },
    },
}



 Comments   
Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'iwysiu', 'email': 'isabella.siu@10gen.com', 'username': 'iwysiu'}

Message: GODRIVER-1363 fix indexes for writeErrors with batches and filter ret… (#362)
Branch: release/1.3
https://github.com/mongodb/mongo-go-driver/commit/a4e103cc4b50655c232ab5e4f98b3a2310ae6caa

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'iwysiu', 'email': 'isabella.siu@10gen.com', 'username': 'iwysiu'}

Message: GODRIVER-1363 fix indexes for writeErrors with batches and filter ret… (#362)
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/c814cfb676dc29a6ad6171e138510ee88c51983f

Comment by Divjot Arora (Inactive) [ 23/Mar/20 ]

joel@sternenbauer.com Thanks for the explanation. Because this would actually change the values returned to the user, it seems risky to introduce this in a patch release. However, we do understand that this has caused you issues and we'd like to address it. We'll be doing this work in the 1.4.0 release cycle. The 1.4.0 driver release will correspond to the MongoDB 4.4 server release.

– Divjot

Comment by Joël Gähwiler [ 19/Mar/20 ]

As discussed with Divjot Arora via email I'm adding some further explanation here to bump the issue.

The problem can be tracked down to the following code:
https://github.com/mongodb/mongo-go-driver/blob/master/mongo/collection.go#L247

As described in the issue, the result containing the array of inserted ids is built before the command is issued and then returned without taking into account potential write errors. The correct behaviour would be to remove ids from the array for which a write error has been returned.

I had two separate scenarios affecting production, the first made me report the issue, the second to get in contact with Divjot. In both situations the code was  inserting documents and expecting some of the inserts to fail due do unique index constraints. By looking at the inserted ids array, the code wanted to figure out which documents have been inserted and which have not. But since the current implementation wrongfully reports all documents as inserted, the logic that handles the failed inserts would never be run to correct the situation in some other way.

I'm sure other users rely on the correctness of this as well. They are not necessarily looking at the ids, but I assume there are quite a few programs that actually use the array to count the number of documents successfully inserted. However, if there was an error the reported number will be wrong...

Again, I think this behavior of the client is a serious bug und should be fixed ASAP. Thanks!

Comment by Joël Gähwiler [ 14/Dec/19 ]

@esha.bhargava Maybe this should be labelled as a "Bug". In my eyes it's a rather serious bug that the client reports documents to be inserted that are not.

Generated at Thu Feb 08 08:36:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.