[SERVER-15434] Bulk insert misses some data Created: 29/Sep/14  Updated: 10/Dec/14  Resolved: 29/Sep/14

Status: Closed
Project: Core Server
Component/s: Write Ops
Affects Version/s: 2.6.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Shamir Shakir [X] Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File test.js    
Operating System: ALL
Steps To Reproduce:

The exact reproduction of this bug is attached in a .js file. The file contains all data and operations that caused the problem. Here are some brief,

  • In the bulk list there are *125* data.
  • Of them *113* unique _id data which should be inserted.
  • Individual write ops insert all 113 data
  • bulk insert 104 data (don't know why). 9 doc missing.
  • Pls check the attached file to get more info
Participants:

 Comments   
Comment by Shamir Shakir [X] [ 29/Sep/14 ]

Thank you very much!

Comment by Eliot Horowitz (Inactive) [ 29/Sep/14 ]

See inset docs here http://api.mongodb.org/python/current/api/pymongo/collection.html

Comment by Shamir Shakir [X] [ 29/Sep/14 ]

How to pass this boolean param in pymongo?

collection.insert(data, True)

seems to be not working.

Comment by Eliot Horowitz (Inactive) [ 29/Sep/14 ]

The bulk insert operation stops at the first error by default.
If you look at the response from the bulk insert, you'll see it failed:

{
	"writeErrors" : [
		{
			"index" : 104,
			"code" : 11000,
			"errmsg" : "insertDocument :: caused by :: 11000 E11000 duplicate key error index: test.batch_test.$_id_ dup key: { : \"http://www.bangladeshdir.com/type/health\" }",
			"op" : {
				"priority" : 0.1,
				"seen" : false,
				"_id" : "http://www.bangladeshdir.com/type/health"
			}
		}
	],
	"writeConcernErrors" : [ ],
	"nInserted" : 104,
	"nUpserted" : 0,
	"nMatched" : 0,
	"nModified" : 0,
	"nRemoved" : 0,
	"upserted" : [ ]
}

You can pass it an option to continue on error and then return a list of errors rather than stopping if you want.

db.batch_test.insert(data,true)

Generated at Thu Feb 08 03:38:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.