Details
-
Bug
-
Resolution: Cannot Reproduce
-
Major - P3
-
None
-
1.6.1
-
None
-
None
-
EC2 / Ubuntu 10.04
-
Linux
Description
Problem:
I was testing a data load script. During the testing I ran out of disk space.
From the mongo client, I could still query and get results
> db.blogs.count()
57389495
> db.blogs.find()
However, after I performed a collection.drop() it appeared that the mongod process died
> db.blogs.drop()
true
> use scaleout
switched to db scaleout
> for (i=0; i < 10; i++) { db.blogs.insert(
); }
dbconnectionpool: connect failed 10.242.221.155:27000 : couldn't connect to server 10.242.221.155:27000}
Looking at the log file I see the following assert
Tue Aug 31 00:04:18 [conn15] Caught Assertion in insert , continuing
Tue Aug 31 00:04:18 [conn15] insert scaleout.blogs exception 0 assertion db/rec.h:126 22ms
Tue Aug 31 00:04:18 [conn3] end connection 10.242.221.155:51326
Tue Aug 31 00:04:18 [conn15] got request after shutdown()
Tue Aug 31 00:04:18 [conn1] end connection 127.0.0.1:38079
Tue Aug 31 00:04:18 [conn17] got request after shutdown()
Tue Aug 31 00:04:18 [conn16] got request after shutdown()
Tue Aug 31 00:04:18 [conn19] end connection 10.194.171.159:42089
Tue Aug 31 00:04:18 [conn18] end connection 10.194.171.159:42088
Tue Aug 31 00:04:18 [conn20] end connection 10.194.171.159:42090
Tue Aug 31 00:04:18 [initandlisten] now exiting
Tue Aug 31 00:04:18 dbexit: ; exiting immediately
Tue Aug 31 00:04:19 [conn4] end connection 10.194.102.4:44198
Tue Aug 31 00:04:20 Got signal: 11 (Segmentation fault).
Tue Aug 31 00:04:20 Backtrace:
0x813619 0x7f19d59d9af0 0x52a535 0x6f61fd 0x6f7b11 0x816220 0x829370 0x7f19d64dc9ca 0x7f19d5a8c69d
/var/opt/mongodb/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x813619]
/lib/libc.so.6(+0x33af0) [0x7f19d59d9af0]
/var/opt/mongodb/bin/mongod(_ZN5mongo9MongoFile13closeAllFilesERSt18basic_stringstreamIcSt11char_traitsIcESaIcEE+0xa5) [0x52a535]
/var/opt/mongodb/bin/mongod(_ZN5mongo8shutdownEv+0x39d) [0x6f61fd]
/var/opt/mongodb/bin/mongod(_ZN5mongo6dbexitENS_8ExitCodeEPKc+0x201) [0x6f7b11]
/var/opt/mongodb/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x1060) [0x816220]
/var/opt/mongodb/bin/mongod(thread_proxy+0x80) [0x829370]
/lib/libpthread.so.0(+0x69ca) [0x7f19d64dc9ca]
/lib/libc.so.6(clone+0x6d) [0x7f19d5a8c69d]
Reproduce:
- create a DB with limited free space (say 1GB)
- insert into the DB and fill up the available disk space
- once the DB is out of space, do a db.collection.drop()
Workaround:
- allocate more disk space
- restart the db with the "--repair" flag
Business Case:
- Reliability
- User Expereince