[SERVER-18967] Failed to start MongoD after hitting out of disk space Created: 14/Jun/15  Updated: 17/Jun/15  Resolved: 17/Jun/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Eitan Klein Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: 32qa
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-18025 Option to allocate disk space headroo... Closed
Operating System: ALL
Steps To Reproduce:

1) hammer.mongo - cap-collection load

Participants:

 Description   

Environment:

db version v3.1.5-pre-
Single node mongod on a physical machine (windows)

Mongod terminated with no disk space error in unexpected scenario (on running stress w/ cap collection).

Since I didn’t expect this to happen, I tried to review the collection stat() information and restart mongod (no incoming traffic to the machine)

I received the error below, which get me to a situation that I can't restart the service in order to perform any maintenance w/o an external drive and etc.

2015-06-14T07:07:19.575-0400 W -        [initandlisten] Detected unclean shutdown - d:\data\db1\mongod.lock is not empty.
2015-06-14T07:07:19.575-0400 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2015-06-14T07:07:19.575-0400 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-06-14T07:07:25.737-0400 E STORAGE  [initandlisten] WiredTiger (-28888) [1434280045:737588][4744:140704081122080], WT_SESSION.checkpoint: journal/WiredTigerLog.0000002677 SetEndOfFile error: There is not enough space on the disk.
 
ÔT ½
2015-06-14T07:07:25.737-0400 E STORAGE  [initandlisten] WiredTiger (0) [1434280045:737588][4744:140704081122080], WT_SESSION.close: scratch buffer allocated and never discarded
2015-06-14T07:07:27.186-0400 E STORAGE  [initandlisten] WiredTiger (-28888) [1434280047:186672][4744:140704081122080], eviction-server: journal/WiredTigerLog.0000002677 SetEndOfFile error: There is not enough space on the disk.
 
¹¥
2015-06-14T07:07:27.191-0400 I -        [initandlisten] Assertion: 28595:-28888: There is not enough space on the disk.
 
 
2015-06-14T07:07:27.193-0400 I STORAGE  [initandlisten] exception in initAndListen: 28595 -28888: There is not enough space on the disk.
 
, terminating
2015-06-14T07:07:27.193-0400 I CONTROL  [initandlisten] dbexit:  rc: 100



 Comments   
Comment by Daniel Pasette (Inactive) [ 17/Jun/15 ]

Closing this as a duplicate of SERVER-18025.
Moving to directoryperdb by default for WiredTiger (SERVER-18963) will help ameliorate the problem, but there is simply no way to restart WiredTiger without replaying the journal, and if the main volume is out of space, it simply won't be able to restart.

Generated at Thu Feb 08 03:49:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.