<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:19:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-28991] Eliminate having individual threads in resmoke.py for flushing log output to logkeeper</title>
                <link>https://jira.mongodb.org/browse/SERVER-28991</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently, for each child process spawned by resmoke.py (e.g. a mongo shell running a test), resmoke.py starts a &lt;tt&gt;LoggerPipe&lt;/tt&gt; thread to asynchronously read from the child process&apos;s output pipe. (One &lt;tt&gt;LoggerPipe&lt;/tt&gt; thread is created for the stdout of the child process and another is created for the stderr.) When resmoke.py is  invoked with &lt;tt&gt;&amp;#45;&amp;#45;log=buildlogger&lt;/tt&gt;, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.6/buildscripts/resmokelib/logging/handlers.py#L64-L70&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the &lt;tt&gt;BufferedHandler&lt;/tt&gt; logging handler causes an &lt;tt&gt;AlarmClock&lt;/tt&gt; thread to be started&lt;/a&gt;. The &lt;tt&gt;AlarmClock&lt;/tt&gt; thread is responsible for periodically sending logs to logkeeper via a POST request either when 2000 lines of output have been produced or when 10 seconds have passed. This means for each test that runs, there are typically 3 threads created for that child process (we don&apos;t expect anything to ever be written to stderr).&lt;/p&gt;

&lt;p&gt;To avoid slowing down test execution when running tests in Evergreen, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.6/buildscripts/resmokelib/testing/report.py#L133-L135&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;joining the &lt;tt&gt;AlarmClock&lt;/tt&gt; thread is deferred until later&lt;/a&gt;. The &lt;tt&gt;AlarmClock&lt;/tt&gt; thread is joined either by the &lt;tt&gt;FlushThread&lt;/tt&gt; or by Python&apos;s &lt;tt&gt;atexit&lt;/tt&gt; handler in the &lt;tt&gt;logging&lt;/tt&gt; package, so it is possible for a significant number of these &lt;tt&gt;AlarmClock&lt;/tt&gt; threads to accumulate is sending POST requests to logkeeper is taking a while. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=yves.duhem&quot; class=&quot;user-hover&quot; rel=&quot;yves.duhem&quot;&gt;yves.duhem&lt;/a&gt; has found through empirical testing that thread contention appers to be responsible for why running the &lt;tt&gt;replica_sets_jscore_passthrough_WT&lt;/tt&gt; task takes longer on Solaris although it isn&apos;t understood why similar behavior is &lt;b&gt;not&lt;/b&gt; observed in the &lt;tt&gt;replica_sets_jscore_passthrough&lt;/tt&gt; task. Additionally, most of these &lt;tt&gt;AlarmClock&lt;/tt&gt; threads should be idle since they are waiting for 10 seconds to pass before doing any work after the test finishes.&lt;/p&gt;

&lt;p&gt;As a way to reduce the number of threads in resmoke.py, we can change the logging model to make use of &lt;a href=&quot;https://docs.python.org/2/library/sched.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Python&apos;s &lt;tt&gt;sched&lt;/tt&gt; module&lt;/a&gt; so that all POST requests are scheduled on a single thread. There is some risk that by making it no longer possible for resmoke.py to send concurrent requests to logkeeper that resmoke.py can fall behind should getting a response from logkeeper take a long time (e.g. on the order of multiple seconds).&lt;/p&gt;</description>
                <environment></environment>
        <key id="377927">SERVER-28991</key>
            <summary>Eliminate having individual threads in resmoke.py for flushing log output to logkeeper</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="max.hirschhorn@mongodb.com">Max Hirschhorn</assignee>
                                    <reporter username="max.hirschhorn@mongodb.com">Max Hirschhorn</reporter>
                        <labels>
                    </labels>
                <created>Wed, 26 Apr 2017 21:07:48 +0000</created>
                <updated>Mon, 30 Oct 2023 23:17:10 +0000</updated>
                            <resolved>Tue, 16 May 2017 16:11:37 +0000</resolved>
                                                    <fixVersion>3.4.7</fixVersion>
                    <fixVersion>3.5.8</fixVersion>
                                    <component>Testing Infrastructure</component>
                                        <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="1616824" author="xgen-internal-githook" created="Fri, 7 Jul 2017 23:27:36 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;visemet&apos;, u&apos;name&apos;: u&apos;Max Hirschhorn&apos;, u&apos;email&apos;: u&apos;max.hirschhorn@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-28991&quot; title=&quot;Eliminate having individual threads in resmoke.py for flushing log output to logkeeper&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-28991&quot;&gt;&lt;del&gt;SERVER-28991&lt;/del&gt;&lt;/a&gt; Eliminate having an AlarmClock thread for each test.&lt;/p&gt;

&lt;p&gt;Changes resmoke.py to use its FlushThread for managing all calls to&lt;br/&gt;
flush() buildlogger handlers.&lt;/p&gt;

&lt;p&gt;(cherry picked from commit a4d2c52dbabc323f752776c7c86ac1d8a60187cb)&lt;br/&gt;
Branch: v3.4&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/4a7a14dac51d112bb235717882c0458668c1332b&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/4a7a14dac51d112bb235717882c0458668c1332b&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1572415" author="xgen-internal-githook" created="Tue, 16 May 2017 16:10:49 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;visemet&apos;, u&apos;name&apos;: u&apos;Max Hirschhorn&apos;, u&apos;email&apos;: u&apos;max.hirschhorn@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-28991&quot; title=&quot;Eliminate having individual threads in resmoke.py for flushing log output to logkeeper&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-28991&quot;&gt;&lt;del&gt;SERVER-28991&lt;/del&gt;&lt;/a&gt; Eliminate having an AlarmClock thread for each test.&lt;/p&gt;

&lt;p&gt;Changes resmoke.py to use its FlushThread for managing all calls to&lt;br/&gt;
flush() buildlogger handlers.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a4d2c52dbabc323f752776c7c86ac1d8a60187cb&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a4d2c52dbabc323f752776c7c86ac1d8a60187cb&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="731835">SERVER-40514</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="14340"><![CDATA[v3.4]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 16 May 2017 16:10:49 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 31 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 31 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>0.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht6gvz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hras8v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1595">TIG 2017-05-08</customfieldvalue>
    <customfieldvalue id="1711">TIG 2017-05-29</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs49sv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>