<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:56:35 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-21191] Mongo node (Primary) failed suddenly</title>
                <link>https://jira.mongodb.org/browse/SERVER-21191</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I have one MongoDB Cluster (using ReplicaSet) with:&lt;/p&gt;

&lt;p&gt;1 Mongo Primary (3.0.5 - 100GB OPLOG)&lt;br/&gt;
1 Mongo Secondary (3.0.5 - 100GB OPLOG)&lt;br/&gt;
1 Mongo Arbiter (3.0.5 - no journal)&lt;/p&gt;

&lt;p&gt;All servers hosted on Ubuntu 14.04 (Trusty).&lt;/p&gt;

&lt;p&gt;This cluster is used for reading and heavy written (many per second, after all is a datasource). Every write uses REPLICA_ACKNOWLEDGED WriteConcern (w = 2) (to prevent desynchronization).&lt;/p&gt;

&lt;p&gt;Today, at 11AM my primary node failed and I&apos;m not seeing no reason to this...&lt;/p&gt;

&lt;p&gt;Here is the event log (with 2 operations beyond the error, one before and (even after abort) one after) (archive first.log - text is too long).&lt;/p&gt;

&lt;p&gt;After that, my re-initialization script get mongo node only again, and some operations took place smoothly, until this error (archive second.log - text is too long).&lt;/p&gt;

&lt;p&gt;After that, all subsequent startup attempts failed with the following error (archive third.log - text is too long).&lt;/p&gt;

&lt;p&gt;After these failures, I tried to start mongo with --repair parameter, but it gave me WT_PANIC too.&lt;/p&gt;

&lt;p&gt;After that, I upgrade my mongo node to 3.0.7 and start --repair again (which is running so far).&lt;/p&gt;

&lt;p&gt;I know you (jira contributors) don&apos;t have nothing with this, but this error appears to me for the second time(WT_PANIC: WiredTiger library panic). First you said this problem was solved in a custom branch (created with my previous issue) and then attached to the next version.&lt;/p&gt;

&lt;p&gt;It&apos;s hard to trust in mongo for production environments after these errors...&lt;/p&gt;

&lt;p&gt;Can someone help me find the possible cause of these errors mentioned above? I really need to know the cause and ways to avoid them...&lt;/p&gt;

&lt;p&gt;Obs.:&lt;/p&gt;

&lt;p&gt;I attach some pictures of server disk usage. The picture with name 1.png is from the time of first occurrence. The picture with name 2.png is from all day window (my node is repairing since 4PM).&lt;/p&gt;</description>
                <environment></environment>
        <key id="236713">SERVER-21191</key>
            <summary>Mongo node (Primary) failed suddenly</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kelsey.schubert@mongodb.com">Kelsey Schubert</assignee>
                                    <reporter username="lucasoares">Lucas</reporter>
                        <labels>
                    </labels>
                <created>Thu, 29 Oct 2015 00:36:03 +0000</created>
                <updated>Mon, 28 Mar 2016 17:56:57 +0000</updated>
                            <resolved>Mon, 28 Mar 2016 17:56:57 +0000</resolved>
                                                                    <component>Admin</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="1216958" author="ramon.fernandez" created="Mon, 28 Mar 2016 17:56:48 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lucasoares&quot; class=&quot;user-hover&quot; rel=&quot;lucasoares&quot;&gt;lucasoares&lt;/a&gt;, we haven&apos;t heard back from you for some time, so we&apos;re going to close this ticket for the time being. If this is still an issue for you, or if you run into a server bug, please open a new ticket. For MongoDB-related support discussion please post on the &lt;a href=&quot;http://groups.google.com/group/mongodb-user&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;mongodb-user group&lt;/a&gt; or &lt;a href=&quot;http://stackoverflow.com/questions/tagged/mongodb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Stack Overflow with the &lt;tt&gt;mongodb&lt;/tt&gt; tag&lt;/a&gt;, where your question will reach a larger audience.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1193808" author="thomas.schubert" created="Fri, 4 Mar 2016 23:22:45 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lucasoares&quot; class=&quot;user-hover&quot; rel=&quot;lucasoares&quot;&gt;lucasoares&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Sorry this ticket slipped through the cracks, is this still an issue for you?&lt;/p&gt;

&lt;p&gt;The logs that you have provided indicate that the server encountered data corruption and shutdown to preserve the integrity of the rest of the dataset. It is likely that the data corruption was the result of faulty disk drives or power failures. However, identifying the exact cause can be challenging without a clear reproduction. If you are continuing to experience issues with data corruption we would recommend a thorough integrity check of your disk drives.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1075476" author="lucasoares" created="Fri, 30 Oct 2015 17:09:09 +0000"  >&lt;p&gt;About the log, exactly!! There is no more log info (except a lot of updates, insertions and finds) in first log. I searched for the last two thousand lines and nothing (log file have 15G =/)&lt;/p&gt;

&lt;p&gt;About the validate command, no, I don&apos;t.. I will wait for repair (or if fails, rsync), to get a replica with at least 2 nodes again. But the problem is: why my server shutdown for the first time? (first.log)&lt;/p&gt;

&lt;p&gt;My data in mongo are separated into many collections (300+). As would be the validation in this case?&lt;/p&gt;</comment>
                            <comment id="1074615" author="ramon.fernandez" created="Thu, 29 Oct 2015 20:23:40 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lucasoares&quot; class=&quot;user-hover&quot; rel=&quot;lucasoares&quot;&gt;lucasoares&lt;/a&gt;, I looked at other issues reported by you and I found &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-19293&quot; title=&quot;Unable to repair corrupted data after a server-crash (using wiredTiger)&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-19293&quot;&gt;&lt;del&gt;SERVER-19293&lt;/del&gt;&lt;/a&gt;. If the data files for the primary node in this ticket were ever part of a 3.0.2/3.0.4 node that suffered an unclean shutdown it could be the case that what you&apos;re seeing here is the effect of such shutdown.&lt;/p&gt;

&lt;p&gt;Did you run ever run &lt;a href=&quot;https://docs.mongodb.org/manual/reference/command/validate/#validate&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;validate()&lt;/tt&gt;&lt;/a&gt; on your data after the unclean shutdown with 3.0.2/3.0.4?&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                            <comment id="1074599" author="ramon.fernandez" created="Thu, 29 Oct 2015 20:15:46 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lucasoares&quot; class=&quot;user-hover&quot; rel=&quot;lucasoares&quot;&gt;lucasoares&lt;/a&gt;, unfortunately the log snippet uploaded in &lt;tt&gt;first.log&lt;/tt&gt; does not contain enough information, could you please upload the full logs for this node so we can investigate what happened?&lt;/p&gt;

&lt;p&gt;I took a quick look at &lt;tt&gt;second.log&lt;/tt&gt; and &lt;tt&gt;third.log&lt;/tt&gt; files, and I believe the messages in them can be explained by the failure in &lt;tt&gt;first.log&lt;/tt&gt;, so I think we should look at a full log for that node first.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
Ram&#243;n.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="95983" name="1.png" size="106507" author="lucasoares" created="Thu, 29 Oct 2015 00:36:03 +0000"/>
                            <attachment id="95984" name="2.png" size="60106" author="lucasoares" created="Thu, 29 Oct 2015 00:36:03 +0000"/>
                            <attachment id="95985" name="first.log" size="6245" author="lucasoares" created="Thu, 29 Oct 2015 00:36:03 +0000"/>
                            <attachment id="95986" name="second.log" size="6018" author="lucasoares" created="Thu, 29 Oct 2015 00:36:03 +0000"/>
                            <attachment id="95987" name="third.log" size="6590" author="lucasoares" created="Thu, 29 Oct 2015 00:36:03 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 29 Oct 2015 20:15:46 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 46 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 46 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>lucasoares</customfieldvalue>
            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrkqe7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hse0db:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[kelsey.schubert@mongodb.com]]></customfieldvalue>
    

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrllsf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>