-
Type: Bug
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Environment:Mac OS X, Linux, 64-bit
During insert of multiple docs whose individual size close to 4MB limit, the bulk insert fails with AutoReconnect exception. To reproduce a bug, split any text file whose size more then 4MB into chunks, e.g. split -b 3m my_text_file.txt (here size of 1 chunk is 3MB). This will create files: xaa, xab, etc. Then you may use the following stand-alone code with xaa file to reproduce the problem
import sys
from pymongo import Connection
db = Connection()
coll = db['db']['test']
data = open('xaa', 'r').read()
print "Size of data", sys.getsizeof(data)
rec =
coll.remove({})
coll.insert(rec)
print "Number of docs:", coll.find().count()
coll.remove({})
docs = [
for k in range(0,10)]
for rec in docs:
coll.insert(rec)
print "Number of docs:", coll.find().count()
coll.remove({})
coll.insert(docs)
print "Number of docs:", coll.find().count()
This code inserts all 10 docs one by one, while raise the following exception during bulk insert:
Traceback (most recent call last):
File "test_bulk.py", line 23, in <module>
coll.insert(docs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/collection.py", line 245, in insert
message.insert(self.__full_name, docs, check_keys, safe), safe)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/pymongo/connection.py", line 574, in _send_message
raise AutoReconnect(str(e))
pymongo.errors.AutoReconnect: [Errno 32] Broken pipe