<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:15:25 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-7725] File allocation leads to full disk, e4fs allocation error</title>
                <link>https://jira.mongodb.org/browse/SERVER-7725</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;On November 3, mongo1, a hidden member of the replica set had mongod go unresponsive.  Examining system logs, we found thousands of messages like:&lt;/p&gt;

&lt;p&gt;Nov  3 17:20:47 276036-mongo1 kernel: EXT4-fs: Can&apos;t allocate: Allocation context details:&lt;br/&gt;
Nov  3 17:20:47 276036-mongo1 kernel: EXT4-fs: status 1 flags 3104&lt;br/&gt;
Nov  3 17:20:47 276036-mongo1 kernel: EXT4-fs: orig 1598/525/1@477864, goal 1598/515/2048@477184, best 0/0/0@477864 cr 3&lt;br/&gt;
Nov  3 17:20:47 276036-mongo1 kernel: EXT4-fs: 0 scanned, 0 found&lt;/p&gt;

&lt;p&gt;The only EXT4 filesystem on the server is the mongo partition, and mongod is the sole process using it.  The only correllating event we could find in mongo logs was:&lt;/p&gt;

&lt;p&gt;Sat Nov  3 17:18:48 &lt;span class=&quot;error&quot;&gt;&amp;#91;FileAllocator&amp;#93;&lt;/span&gt; allocating new datafile /var/lib/mongo/enlighten_production/enlighten_production.93, filling with zeroes...&lt;/p&gt;

&lt;p&gt;On other FileAllocator events, an &#8216;allocation complete&#8217; message was logged as well.  This one had no matching completion message.  Eventually mongod failed:&lt;/p&gt;

&lt;p&gt;Sat Nov  3 17:25:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;journal&amp;#93;&lt;/span&gt; LogFile::synchronousAppend failed with 139264 bytes unwritten out of 139264 bytes;  b=0x2adec2402000 errno:28 No space left on device&lt;br/&gt;
Sat Nov  3 17:25:41 Got signal: 6 (Aborted).&lt;/p&gt;

&lt;p&gt;Sat Nov  3 17:25:41 Backtrace:&lt;br/&gt;
0xa95ce9 0x3f322302d0 0x3f32230265 0x3f32231d10 0x75c13d 0x777da5 0x777fde 0x762f1c 0x76376d 0x763a8d 0x76436b 0xaabca0 0x3f3360673d 0x3f322d44bd&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x3a9) &lt;span class=&quot;error&quot;&gt;&amp;#91;0xa95ce9&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib64/libc.so.6 &lt;span class=&quot;error&quot;&gt;&amp;#91;0x3f322302d0&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib64/libc.so.6(gsignal+0x35) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x3f32230265&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib64/libc.so.6(abort+0x110) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x3f32231d10&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo7LogFile17synchronousAppendEPKvm+0x12d) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x75c13d&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo3dur7Journal7journalERKNS0_11JSectHeaderERKNS_14AlignedBuilderE+0x1e5) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x777da5&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo3dur14WRITETOJOURNALENS0_11JSectHeaderERNS_14AlignedBuilderE+0x4e) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x777fde&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo3dur28_groupCommitWithLimitedLocksEv+0x24c) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x762f1c&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo3dur27groupCommitWithLimitedLocksEv+0x1d) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x76376d&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod &lt;span class=&quot;error&quot;&gt;&amp;#91;0x763a8d&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(_ZN5mongo3dur9durThreadEv+0x10b) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x76436b&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /usr/bin/mongod(thread_proxy+0x80) &lt;span class=&quot;error&quot;&gt;&amp;#91;0xaabca0&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib64/libpthread.so.0 &lt;span class=&quot;error&quot;&gt;&amp;#91;0x3f3360673d&amp;#93;&lt;/span&gt;&lt;br/&gt;
 /lib64/libc.so.6(clone+0x6d) &lt;span class=&quot;error&quot;&gt;&amp;#91;0x3f322d44bd&amp;#93;&lt;/span&gt;&lt;br/&gt;
Sat Nov  3 17:25:41 Invalid access at address: 0&lt;br/&gt;
Sat Nov  3 17:25:41 Got signal: 11 (Segmentation fault).&lt;/p&gt;

&lt;p&gt;The server was rebooted, and mongo was able to recover itself as secondary.  Three days later, the primary server in the set, mongo3 (syslog still had the hostname as cache2), had the same issue:&lt;/p&gt;

&lt;p&gt;Nov  6 10:44:46 421762-cache2 kernel: EXT4-fs: Can&apos;t allocate: Allocation context details:&lt;br/&gt;
Nov  6 10:44:46 421762-cache2 kernel: EXT4-fs: status 1 flags 3104&lt;br/&gt;
Nov  6 10:44:46 421762-cache2 kernel: EXT4-fs: orig 1597/1831/64@300060, goal 1597/1830/2048@299008, best 0/0/0@300060 cr 3&lt;br/&gt;
Nov  6 10:44:46 421762-cache2 kernel: EXT4-fs: 0 scanned, 0 found&lt;/p&gt;


&lt;p&gt;Tue Nov  6 10:44:29 &lt;span class=&quot;error&quot;&gt;&amp;#91;FileAllocator&amp;#93;&lt;/span&gt; allocating new datafile /var/lib/mongo/enlighten_production/enlighten_production.94, filling with zeroes...&lt;br/&gt;
...&lt;br/&gt;
Tue Nov  6 10:46:11 &lt;span class=&quot;error&quot;&gt;&amp;#91;journal&amp;#93;&lt;/span&gt; LogFile::synchronousAppend failed with 114688 bytes unwritten out of 114688 bytes;  b=0x2adccb794000 errno:28 No space left on device&lt;br/&gt;
Tue Nov  6 10:46:11 Got signal: 6 (Aborted).&lt;/p&gt;

&lt;p&gt;In both cases, there was more than 100GB free on the partition at the time.  It looks as though the allocation, or the zeroing, went haywire and filled the whole partition.&lt;/p&gt;</description>
                <environment>uname: Linux 421762-mongo3.enl.enphaseenergy.com 2.6.18-238.45.1.el5 #1 SMP Thu Sep 20 12:19:35 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux&lt;br/&gt;
e4fsprogs version 1.41.12-2.el5&lt;br/&gt;
mongo version mongo-10gen-server-2.0.6-mongodb_1.x86_64&lt;br/&gt;
Hardware: Dell PowerEdge 2970, PowerEdge R710</environment>
        <key id="57054">SERVER-7725</key>
            <summary>File allocation leads to full disk, e4fs allocation error</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="david.hows">David Hows</assignee>
                                    <reporter username="bascott@enphaseenergy.com">Benjamin Abbott-Scott</reporter>
                        <labels>
                    </labels>
                <created>Tue, 20 Nov 2012 18:27:36 +0000</created>
                <updated>Fri, 8 Mar 2013 15:56:11 +0000</updated>
                            <resolved>Sun, 24 Feb 2013 23:16:34 +0000</resolved>
                                    <version>2.0.6</version>
                                                    <component>Stability</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="199875" author="david.hows" created="Tue, 27 Nov 2012 07:20:45 +0000"  >&lt;p&gt;Hi Benjamin,&lt;/p&gt;

&lt;p&gt;From your dmesg output i can see a couple of things worth noting, but first has the system been rebooted since the issue occurred? And has the issue recurred subsequently?&lt;/p&gt;

&lt;p&gt;The devices dm-3 and dm-4 are both running ext4 and appear to be running without journal and are running unchecked. Additionally dm-3 has a mount options which it cannot parse and appears to have failed its barrier check. &lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-3): mounted filesystem without journal&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-3): Unrecognized mount option &quot;discard&quot; or missing value&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-3): Unrecognized mount option &quot;discard&quot; or missing value&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-3): mounted filesystem with writeback data mode&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;JBD: barrier-based sync failed on dm-3-8 - disabling barriers&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-4): warning: mounting unchecked fs, running e2fsck is recommended&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;EXT4-fs (dm-4): mounted filesystem without journal&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Can you run the following command to ID which devices map to which of the vol groups?&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;lvdisplay|awk  &apos;/LV Name/{n=$3} /Block device/{d=$3; sub(&quot;.*:&quot;,&quot;dm-&quot;,d); print d,n;}&apos;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Can you please attach your &lt;tt&gt;/etc/fstab&lt;/tt&gt; file? I would like to see what mount options are being passed to where.&lt;/p&gt;

&lt;p&gt;Finally, can you give some background as to why you are running your filesystems in barrier mode and writeback mode?&lt;/p&gt;

&lt;p&gt;Cheers,&lt;/p&gt;

&lt;p&gt;David&lt;/p&gt;</comment>
                            <comment id="199764" author="bascott@enphaseenergy.com" created="Tue, 27 Nov 2012 04:24:34 +0000"  >&lt;p&gt;mongod log minus connection actions (on the order of 4200/min)&lt;/p&gt;</comment>
                            <comment id="199761" author="bascott@enphaseenergy.com" created="Tue, 27 Nov 2012 04:18:23 +0000"  >&lt;p&gt;Attaching dmesg output.  SMART not available as the drives are presented by the PERC controller, but omsa reports no predicted failures.&lt;/p&gt;

&lt;p&gt;bascott@421762-mongo3 20:10 &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=&quot; class=&quot;user-hover&quot; rel=&quot;&quot;&gt;&lt;/a&gt;$ df -h&lt;br/&gt;
Filesystem            Size  Used Avail Use% Mounted on&lt;br/&gt;
/dev/mapper/201206141135-root&lt;br/&gt;
                       37G  6.4G   29G  19% /&lt;br/&gt;
/dev/mapper/201206141135-temp&lt;br/&gt;
                      2.0G   85M  1.8G   5% /tmp&lt;br/&gt;
/dev/sda1             244M   27M  205M  12% /boot&lt;br/&gt;
tmpfs                  24G     0   24G   0% /dev/shm&lt;br/&gt;
/dev/mapper/vg2-vg2_mongo&lt;br/&gt;
                      553G  415G  111G  80% /var/lib/mongo&lt;br/&gt;
/dev/mapper/vg1-vg1_backups&lt;br/&gt;
                       58G   59M   55G   1% /var/dump/mongo&lt;br/&gt;
bascott@421762-mongo3 20:10 &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=&quot; class=&quot;user-hover&quot; rel=&quot;&quot;&gt;&lt;/a&gt;$ mount&lt;br/&gt;
/dev/mapper/201206141135-root on / type ext3 (rw)&lt;br/&gt;
proc on /proc type proc (rw)&lt;br/&gt;
sysfs on /sys type sysfs (rw)&lt;br/&gt;
devpts on /dev/pts type devpts (rw,gid=5,mode=620)&lt;br/&gt;
/dev/mapper/201206141135-temp on /tmp type ext3 (rw,noexec,nosuid,nodev)&lt;br/&gt;
/dev/sda1 on /boot type ext3 (rw)&lt;br/&gt;
tmpfs on /dev/shm type tmpfs (rw,noexec,nosuid,nodev)&lt;br/&gt;
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)&lt;br/&gt;
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)&lt;br/&gt;
/dev/mapper/vg2-vg2_mongo on /var/lib/mongo type ext4 (rw,noatime,nodiratime,data=writeback)&lt;br/&gt;
/dev/mapper/vg1-vg1_backups on /var/dump/mongo type ext4 (rw)&lt;/p&gt;

&lt;p&gt;sudo /usr/sbin/vgdisplay -v vg2&lt;br/&gt;
    Using volume group(s) on command line&lt;br/&gt;
    Finding volume group &quot;vg2&quot;&lt;br/&gt;
  &amp;#8212; Volume group &amp;#8212;&lt;br/&gt;
  VG Name               vg2&lt;br/&gt;
  System ID&lt;br/&gt;
  Format                lvm2&lt;br/&gt;
  Metadata Areas        1&lt;br/&gt;
  Metadata Sequence No  3&lt;br/&gt;
  VG Access             read/write&lt;br/&gt;
  VG Status             resizable&lt;br/&gt;
  MAX LV                0&lt;br/&gt;
  Cur LV                1&lt;br/&gt;
  Open LV               1&lt;br/&gt;
  Max PV                0&lt;br/&gt;
  Cur PV                1&lt;br/&gt;
  Act PV                1&lt;br/&gt;
  VG Size               561.57 GB&lt;br/&gt;
  PE Size               4.00 MB&lt;br/&gt;
  Total PE              143761&lt;br/&gt;
  Alloc PE / Size       143761 / 561.57 GB&lt;br/&gt;
  Free  PE / Size       0 / 0&lt;br/&gt;
  VG UUID               zFk7tA-P01a-J5NS-JQzM-5C4n-5cSx-kfjAMz&lt;/p&gt;

&lt;p&gt;  &amp;#8212; Logical volume &amp;#8212;&lt;br/&gt;
  LV Name                /dev/vg2/vg2_mongo&lt;br/&gt;
  VG Name                vg2&lt;br/&gt;
  LV UUID                6giOHJ-ijqj-kqsK-ZZsK-Rw1V-wHeM-MtmvL5&lt;br/&gt;
  LV Write Access        read/write&lt;br/&gt;
  LV Status              available&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;open                 1&lt;br/&gt;
  LV Size                561.57 GB&lt;br/&gt;
  Current LE             143761&lt;br/&gt;
  Segments               1&lt;br/&gt;
  Allocation             inherit&lt;br/&gt;
  Read ahead sectors     auto&lt;/li&gt;
&lt;/ol&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;currently set to     256&lt;br/&gt;
  Block device           253:3&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;  &amp;#8212; Physical volumes &amp;#8212;&lt;br/&gt;
  PV Name               /dev/sda5&lt;br/&gt;
  PV UUID               PwYz1m-5NFH-zqJW-e9p6-M37C-nrmK-p8HYQR&lt;br/&gt;
  PV Status             allocatable&lt;br/&gt;
  Total PE / Free PE    143761 / 0&lt;/p&gt;</comment>
                            <comment id="196319" author="david.hows" created="Thu, 22 Nov 2012 04:42:49 +0000"  >&lt;p&gt;Hi Benjamin,&lt;/p&gt;

&lt;p&gt;This particular error comes from the Linux kernel itself and is indicative of some form of error working with the ext4 device. &lt;/p&gt;

&lt;p&gt;Can you please attach log files mongod instance in question, as I would like to see what it was doing beforehand.&lt;/p&gt;

&lt;p&gt;Can you also provide:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;The output of &lt;tt&gt;df -h&lt;/tt&gt;. This gives us an idea of your disk layout.&lt;/li&gt;
	&lt;li&gt;The output of &lt;tt&gt;mount&lt;/tt&gt;. This will show us the mount options used for your fileshare&lt;/li&gt;
	&lt;li&gt;The full output of &lt;tt&gt;dmesg&lt;/tt&gt;. This is the kernel log, which may have some indicators as to why there were EXT4 issues.&lt;/li&gt;
	&lt;li&gt;If possible can you provide the output of &lt;tt&gt;smartctl -H /dev/XXXX&lt;/tt&gt; Where /dev/XXXX is the ext4 device mongo is mounted on. This will do a disk health check on a SMART enabled drive.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;It may also be worthwhile you&apos;re looking to see if RedHat as any information on this issue.&lt;/p&gt;

&lt;p&gt;Cheers,&lt;/p&gt;

&lt;p&gt;David&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="21958" name="dmesg.txt" size="40910" author="bascott@enphaseenergy.com" created="Tue, 27 Nov 2012 04:18:42 +0000"/>
                            <attachment id="21957" name="dmesg.txt" size="40910" author="bascott@enphaseenergy.com" created="Tue, 27 Nov 2012 04:18:18 +0000"/>
                            <attachment id="21959" name="mongod.log" size="604664" author="bascott@enphaseenergy.com" created="Tue, 27 Nov 2012 04:24:34 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 22 Nov 2012 04:42:49 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 12 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 12 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>bascott@enphaseenergy.com</customfieldvalue>
            <customfieldvalue>david.hows</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrngzr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrkohr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>34423</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Unable to reproduce intentionally&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs7h6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>