<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:17:10 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-70802] Mongod data directory and FTDC files not uploaded as part of timeout</title>
                <link>https://jira.mongodb.org/browse/SERVER-70802</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;&lt;a href=&quot;https://www.mongodb.com/docs/manual/administration/analyzing-mongodb-performance/#full-time-diagnostic-data-capture&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://www.mongodb.com/docs/manual/administration/analyzing-mongodb-performance/#full-time-diagnostic-data-capture&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;It would appear that FTDC data isn&apos;t being collected from a timeout failure.&lt;/p&gt;

&lt;p&gt;A project was undertaken to retrieve FTDC data from a timeout failure, including work done in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46687&quot; title=&quot;Run hang-analyzer from resmoke and integrate with archival&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46687&quot;&gt;&lt;del&gt;SERVER-46687&lt;/del&gt;&lt;/a&gt;. Perhaps this stopped working recently? This error was reported because of trying to solve a BF.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2165295">SERVER-70802</key>
            <summary>Mongod data directory and FTDC files not uploaded as part of timeout</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="juan.gu@mongodb.com">Juan Gu</assignee>
                                    <reporter username="alex.neben@mongodb.com">Alex Neben</reporter>
                        <labels>
                    </labels>
                <created>Mon, 24 Oct 2022 15:24:46 +0000</created>
                <updated>Sun, 29 Oct 2023 21:31:33 +0000</updated>
                            <resolved>Wed, 21 Dec 2022 19:47:40 +0000</resolved>
                                                    <fixVersion>6.3.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5069123" author="xgen-internal-githook" created="Wed, 21 Dec 2022 01:00:30 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Juan Gu&apos;, &apos;email&apos;: &apos;juan.gu@mongodb.com&apos;, &apos;username&apos;: &apos;juangugit&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70802&quot; title=&quot;Mongod data directory and FTDC files not uploaded as part of timeout&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70802&quot;&gt;&lt;del&gt;SERVER-70802&lt;/del&gt;&lt;/a&gt; Ensure data files are uploaded on Evergreen timeout&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6cd0e5c322cf8cc1b24722543a0f7e5604f85ed8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6cd0e5c322cf8cc1b24722543a0f7e5604f85ed8&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="4991225" author="max.hirschhorn@10gen.com" created="Thu, 17 Nov 2022 21:06:06 +0000"  >&lt;p&gt;Juan, Tausif, and I walked through how PM-1569 achieved the hang analyzer causing data files to be uploaded.&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;The &lt;tt&gt;evergreen&lt;/tt&gt; agent triggers &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e332ea00d872740058898541277bf6547774be90/etc/evergreen_yml_components/definitions.yml#L2268-L2272&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the &lt;tt&gt;timeout:&lt;/tt&gt; phase of the Evergreen project configuration&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;&lt;tt&gt;python buildscripts/resmoke.py hang&amp;#45;analyzer&lt;/tt&gt; is called to send a SIGUSR1 to the resmoke process running tests.&lt;/li&gt;
	&lt;li&gt;The SIGUSR1 handler in resmoke &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e332ea00d872740058898541277bf6547774be90/buildscripts/resmokelib/sighandler.py#L156&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;invokes &lt;tt&gt;python buildscripts/resmoke.py hang&amp;#45;analyzer&lt;/tt&gt;&lt;/a&gt; to attach the debugger on all the children, grandchildren, etc. of processes transitively spawned through resmoke.&lt;/li&gt;
	&lt;li&gt;The hang analyzer invokes &lt;tt&gt;gdb&lt;/tt&gt; to capture core dumps of all of the MongoDB processes. See also &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-56167&quot; title=&quot;Guarantee hang analyzer collects core dumps for sharded clusters, at minimum&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-56167&quot;&gt;&lt;del&gt;SERVER-56167&lt;/del&gt;&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;The hang analyzer invokes &lt;tt&gt;gdb&lt;/tt&gt; to capture additional diagnostic information from the live process from all of the MongoDB processes.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/e332ea00d872740058898541277bf6547774be90/buildscripts/resmokelib/hang_analyzer/hang_analyzer.py#L163-L165&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;The hang analyzer sends a SIGKILL to all of the analyzed processes&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;The resmoke job thread detects the test has now completed and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/e332ea00d872740058898541277bf6547774be90/buildscripts/resmokelib/testing/job.py#L201-L203&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;runs archival, if enabled&lt;/a&gt;.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;The fundamental issue is that resmoke archival running depends on the hang analyzer killing the processes. However, step (5) is known to take a long enough time where the &lt;tt&gt;evergreen&lt;/tt&gt; agent abandons the work in the &lt;tt&gt;timeout:&lt;/tt&gt; phase of the Evergreen project configuration after 15 minutes.&lt;/p&gt;

&lt;p&gt;My proposal for &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-70802&quot; title=&quot;Mongod data directory and FTDC files not uploaded as part of timeout&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-70802&quot;&gt;&lt;del&gt;SERVER-70802&lt;/del&gt;&lt;/a&gt; would be to spawn an additional thread inside of hang analyzer to monitor the &lt;tt&gt;gdb&lt;/tt&gt; process in step (5) and to kill the &lt;tt&gt;gdb&lt;/tt&gt; process so the remaining steps complete within the 15 minutes allotted by the &lt;tt&gt;evergreen&lt;/tt&gt; agent. This strategy will leave us with always getting core dumps and always getting data files.&lt;/p&gt;

&lt;p&gt;The longer term outlook I have for collecting diagnostics for hangs is that we rely more on post&amp;#45;processing to get the same information we can from the live process. This has very much been the motivation for me creating &lt;a href=&quot;https://github.com/visemet/gdb-mongodb-server&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/visemet/gdb-mongodb-server&lt;/a&gt; to supplant the &lt;tt&gt;mongodb&amp;#45;dump&amp;#45;locks&lt;/tt&gt; gdb command.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10020">
                    <name>Gantt Dependency</name>
                                            <outwardlinks description="has to be done before">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2227014">SERVER-72613</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                                        </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 24 Oct 2022 22:08:53 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 7 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 7 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>14.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alex.neben@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>juan.gu@mongodb.com</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1f4pb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0xs8w:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1equn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>