<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:11:16 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-6292] MongoD server crashes after inserted about 10GB worth of data with &quot;Assertion failure a &lt;= 512*1024*1024 util/alignedbuilder.cpp&quot;</title>
                <link>https://jira.mongodb.org/browse/SERVER-6292</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I&apos;m running a test, based on gridfs_test.c test_large().&lt;br/&gt;
I modified this test to operate with a 5GB file instead of the standard 3GB file.&lt;br/&gt;
I have also modified the source code of GridFS.c to allow for random writing into a file (I will commit this codes on my fork and request a Pull). The most significant change is that it now does an Update (upsert) of the data, in case the file already existed if overwriting a block.&lt;/p&gt;

&lt;p&gt;If I run the test once with my configuration, which will create the file twice on the DB with a deletion in between, when is trying to create the file on the second part after deleting the first file it will crash. This happens right away, no waits or nothing.&lt;br/&gt;
I know it&apos;s not something functional with the test, because if I bring MongoD up again and run the test again over the same database, it will work.&lt;br/&gt;
The Commit Size seams to grow unstoppably until it hits about 5.4GB and it hovers there. I have limited Working set consumption to 1GB for the sake of stressing the server a little more. I tried also removing this constraint letting mongo consume everything he wants with the same result, so I see no reason to let it eat all of my RAM. (I used this to limit Mongo RAM &lt;a href=&quot;http://captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://captaincodeman.com/2011/02/27/limit-mongodb-memory-use-windows/&lt;/a&gt;)&lt;/p&gt;

&lt;p&gt;I have attached MongoDConsole.JPG. It shows the warning before the server crashes.&lt;br/&gt;
I did put a breakpoint on the place where I&apos;m writing to the DB to let it &quot;sit&quot; for a while, and then let it run again.&lt;br/&gt;
The warning appeared again, as soon as it appeared I put a breakpoint again at the insertion point and let it sit for a while.&lt;br/&gt;
I run some other tests I have, light weight one and saw journal files moving getting cleaned up.&lt;br/&gt;
I removed the breakpoint, and let the code run again and it finished this time.&lt;/p&gt;

&lt;p&gt;See attachment MongoDConsole_3.jpg. That&apos;s after another run. The previous run succeeded, so I started the test again. Got the warning more or less right away, and right after that the Journal processor got rid of a lot of journal files, it seems things stabilized there.&lt;br/&gt;
Something else to mention is that I&apos;m removing the old &quot;File&quot; every time I run the tests, which implies all chunks have to be removed and putting some load on the process.&lt;/p&gt;

&lt;p&gt;On this run. I got the warning again, and I&apos;m letting the system run to see if I can get it to crash.&lt;br/&gt;
Got the warning a second time on the final run, but didn&apos;t crash (MongoDConsole_4JPG.jpg).&lt;br/&gt;
This test, after the interventions finished fine.&lt;/p&gt;

&lt;p&gt;Launched the test again, but this time decided not to intervene and let the program run even if getting the warning. Got it right away after the process started, and a little while after got the kiss of death. Warning and crash (see MongoDConsole_crash.jpg).&lt;/p&gt;

&lt;p&gt;This definitively seems to be a problem with rate of IO.&lt;/p&gt;

&lt;p&gt;I do think MongoD should be smart enough to &quot;throttle&quot; the client who is pushing data, as the client has no way of knowing this is going on at the MongoD side.&lt;/p&gt;
</description>
                <environment>Windows 2008 64 bits R2&lt;br/&gt;
Virtualized over VMWare, 4 virtual CPUs (2 real on host), Intel Xeon E5520 2.27GHz&lt;br/&gt;
6GB of RAM, 8GB of page file&lt;br/&gt;
virtual disk with 20GB of space (lots available at time of crash)</environment>
        <key id="43283">SERVER-6292</key>
            <summary>MongoD server crashes after inserted about 10GB worth of data with &quot;Assertion failure a &lt;= 512*1024*1024 util/alignedbuilder.cpp&quot;</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="4">Incomplete</resolution>
                                        <assignee username="mathias@mongodb.com">Mathias Stearn</assignee>
                                    <reporter username="k2xt">Jose Sebastian Battig</reporter>
                        <labels>
                            <label>SERVER_V2</label>
                            <label>Windows</label>
                            <label>crash</label>
                            <label>insert</label>
                            <label>performance</label>
                            <label>update</label>
                    </labels>
                <created>Tue, 3 Jul 2012 15:29:37 +0000</created>
                <updated>Fri, 8 Mar 2013 15:55:54 +0000</updated>
                            <resolved>Tue, 27 Nov 2012 16:46:49 +0000</resolved>
                                    <version>2.0.6</version>
                                                    <component>GridFS</component>
                    <component>Internal Code</component>
                    <component>Usability</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="139945" author="redbeard0531" created="Thu, 5 Jul 2012 16:21:37 +0000"  >&lt;p&gt;FYI, GridFS intentionally doesn&apos;t support modifying files in-place, so if you plan on doing that I&apos;d advise against using the standard fs.files/fs.chunks as many tools will not be expecting it. That said, this should definitely not cause the error you are seeing.&lt;/p&gt;

&lt;p&gt;In order to narrow down the cause here, do you know which of the four test_* functions is causing the crash?&lt;/p&gt;</comment>
                            <comment id="139304" author="k2xt" created="Tue, 3 Jul 2012 20:11:27 +0000"  >&lt;p&gt;I did a test now launching mongod with option --journalCommitInterval 5 and then using WriteConcern with j = 1, and the performance is more acceptable.&lt;br/&gt;
It looks like when launching mongod with default settings, every I/O operation launched by the driver has to wait until mongo passes thru the data cached in memory to be written to the journal, therefore making any I/O operation take at least 100ms.&lt;br/&gt;
Using this setting at least mongo is trying to flush the data more often, making every I/O with default write concern more acceptble&lt;/p&gt;</comment>
                            <comment id="139223" author="k2xt" created="Tue, 3 Jul 2012 19:00:10 +0000"  >&lt;p&gt;I added the following code as soon as I connect to the server:&lt;/p&gt;

&lt;p&gt;mongo_write_concern_init(&amp;amp;wc);&lt;br/&gt;
wc.j = 1;&lt;br/&gt;
mongo_write_concern_finish(&amp;amp;wc);&lt;br/&gt;
mongo_set_write_concern(conn, &amp;amp;wc);&lt;/p&gt;

&lt;p&gt;And then the problem doesn&apos;t appear anymore. The problem is that general operation of the server seems really slow... &lt;br/&gt;
It will be nice if Mongo server did the throttling automatically as suggested on the issue report without the need to use writeconcerns&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="17907" name="MongoDConsole.JPG" size="106523" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                            <attachment id="17908" name="MongoDConsole_2.JPG" size="233889" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                            <attachment id="17909" name="MongoDConsole_3.JPG" size="214326" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                            <attachment id="17910" name="MongoDConsole_4JPG.JPG" size="113938" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                            <attachment id="17911" name="MongoDConsole_Crash.JPG" size="293335" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                            <attachment id="17912" name="gridfs_test.c" size="10968" author="k2xt" created="Tue, 3 Jul 2012 15:29:37 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 5 Jul 2012 16:21:37 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        11 years, 32 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            11 years, 32 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10022"><![CDATA[Windows]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>k2xt</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrnxl3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrg8lb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8297</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10166" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Tests Written</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10154"><![CDATA[Complete]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkqyf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>