[SERVER-9350] Fatal assertion 13515 on disk full Created: 15/Apr/13 Updated: 15/Jan/15 Resolved: 24/Apr/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Stability |
| Affects Version/s: | 2.4.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | David Gubler | Assignee: | Stennie Steneker (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Configuration: Debian Linux, package from official source:
~# df |
||
| Issue Links: |
|
||||||||
| Operating System: | Linux | ||||||||
| Steps To Reproduce: | I have not experienced the disk full situation on any other MongoDB servers, thus I cannot tell if this problem only happens in this very specific situation or in general. Since this is a production deployment I don't want to do this sort of experiment. |
||||||||
| Participants: | |||||||||
| Description |
|
MongoDB crashed when the disk got full. No suspcious log entries before the one below. dmesg also looks fine. I found several other bug reports relating to full disks, but as far as I can tell they don't apply to the current version. Sun Apr 14 16:25:33.111 [journal] LogFile::synchronousAppend failed with 24576 bytes unwritten out of 24576 bytes; b=0x7fe7703ee000 errno:28 No space left on device |
| Comments |
| Comment by Stennie Steneker (Inactive) [ 24/Apr/13 ] |
|
Hi David, No bother at all .. we appreciate any feedback. Output with a stacktrace is indeed somewhat scary, but this gives us more information on the specific code path that triggered an exception. There is certainly room for improvement and making some of the messaging friendlier. In some common exceptions (such as a secondary too stale to sync or various startup warnings) we have added links to the online documentation. I think this is another case that would benefit from further explanation since the shutdown is expected behaviour. I will check if there is already a feature request for same, and raise one if not. Regards, |
| Comment by David Gubler [ 24/Apr/13 ] |
|
Thanks, Stephen, and sorry for the bother. Unfortunately the output does not look like it is on purpose... I didn't even dare to restart it without removing all the data (and re-syncing to the replica set) because I was afraid of data corruption. |
| Comment by Stennie Steneker (Inactive) [ 24/Apr/13 ] |
|
Hi David, This behaviour is by design: if your server runs out of disk space for journal files, the server process will exit. For more details please see the linked duplicate issue Thanks, |
| Comment by David Gubler [ 15/Apr/13 ] |
|
I forgot. This machine is a XEN VM, the disk image is an LVM volume (hence the /dev/xvda device name) on an SSD, the file system is EXT4 mounted with the discard option. /dev/xvda1 / ext4 rw,noatime,errors=remount-ro,user_xattr,acl,barrier=1,data=ordered,discard 0 0 |