<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:39:17 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[GODRIVER-2750] High goroutine count in connection read/write</title>
                <link>https://jira.mongodb.org/browse/GODRIVER-2750</link>
                <project id="14289" key="GODRIVER">Go Driver</project>
                    <description>&lt;h4&gt;&lt;a name=&quot;Summary&quot;&gt;&lt;/a&gt;Summary&lt;/h4&gt;

&lt;p&gt;&lt;em&gt;Go application experiences&#160;behavior where the application runs okay for a while but then over time gradually begins to consume more CPU until 100% of 1 CPU is consumed and the application mostly stops responding.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The application is reading messages from a collection via a tailable cursor. &#160;As the application increases in CPU usage, the number of goroutines increase:&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;go.mongodb.org/mongo-driver/x/mongo/driver/topology.(*connection).write.func2 N=XX&lt;/em&gt;&lt;br/&gt;
&lt;em&gt;go.mongodb.org/mongo-driver/x/mongo/driver/topology.(*connection).read.func2 N=XX&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;N starts out small - under 10 and then increases to over 700 for both write and read. &#160;I have ran the command `db.currentOp(true)` on the Mongo server to ensure connections are not being leaked and most of the time the application only has 3-5 connections.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;The application is processing the messages one at a time so it should not require a large number of connections to&#160;Mongo.&lt;/em&gt;&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;Pleaseprovidetheversionofthedriver.Ifapplicable%2CpleaseprovidetheMongoDBserverversionandtopology%28standalone%2Creplicaset%2Corshardedcluster%29.&quot;&gt;&lt;/a&gt;Please provide the version of the driver. If applicable, please provide the MongoDB server version and topology (standalone, replica set, or sharded cluster).&lt;/h4&gt;

&lt;p&gt;Mongo version: 5.0.14&lt;/p&gt;

&lt;p&gt;Go version: 1.19.5&lt;/p&gt;

&lt;p&gt;Reproduced on mongo driver versions 1.9.1 and 1.11.1.&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;HowtoReproduce&quot;&gt;&lt;/a&gt;How to Reproduce&lt;/h4&gt;

&lt;p&gt;&lt;em&gt;Do not have a&#160;reproducible test case outside of application.&lt;/em&gt;&lt;/p&gt;
&lt;h4&gt;&lt;a name=&quot;AdditionalBackground&quot;&gt;&lt;/a&gt;Additional Background&lt;/h4&gt;

&lt;p&gt;&lt;em&gt;If problem is in application code, need some guidance on what kind of application code would cause a very high number of connection read/write goroutines but a low number of database connections.&lt;/em&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="2254219">GODRIVER-2750</key>
            <summary>High goroutine count in connection read/write</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="qingyang.hu@mongodb.com">Qingyang Hu</assignee>
                                    <reporter username="luke.mauldin@l3harris.com">Luke Mauldin</reporter>
                        <labels>
                    </labels>
                <created>Mon, 6 Feb 2023 20:22:58 +0000</created>
                <updated>Mon, 1 May 2023 20:48:33 +0000</updated>
                            <resolved>Mon, 1 May 2023 20:48:33 +0000</resolved>
                                    <version>1.9.1</version>
                    <version>1.11.1</version>
                                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="5392170" author="JIRAUSER1268281" created="Mon, 1 May 2023 20:48:00 +0000"  >&lt;p&gt;Hello &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=luke.mauldin%40l3harris.com&quot; class=&quot;user-hover&quot; rel=&quot;luke.mauldin@l3harris.com&quot;&gt;luke.mauldin@l3harris.com&lt;/a&gt;, we appreciate your report and the detailed information.&lt;/p&gt;

&lt;p&gt;Unfortunately, we neither discovered any significant changes around the suspicious code nor were able to reproduce the high goroutine count. Therefore, we are going to close this ticket. However, please feel free to reopen this one or file a new one if you observe the issue again.&lt;/p&gt;

&lt;p&gt;Again, we appreciate your support on MongoDB Go Driver.&lt;/p&gt;</comment>
                            <comment id="5191164" author="luke.mauldin@l3harris.com" created="Sun, 12 Feb 2023 23:38:51 +0000"  >&lt;ol&gt;
	&lt;li&gt;&#160;This behavior is only triggered in certain application batch processing when a lot of messages are processed by one instance of the application. &#160;Unfortunately I can&apos;t say the first mongo driver that started experiencing the issue. &#160;I did testing back to 1.9.1 and I experienced the issue there as well. &#160;We are also using Go 1.19.5, not sure if that matters. &#160;I know previous versions of the application were built with Go 1.17 and 1.18 and they did not exhibit the bug.&lt;/li&gt;
	&lt;li&gt;I updated the current code to be very close to &lt;a href=&quot;https://github.com/mongodb/mongo-go-driver/blob/0e5ae89f1baf3d6edca093467ffd1f870b441add/mongo/crud_examples_test.go#L908-L929&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo-go-driver/blob/0e5ae89f1baf3d6edca093467ffd1f870b441add/mongo/crud_examples_test.go#L908-L929&lt;/a&gt;. The previous code was similar but just used cursor.Next() and did not check for RemainingBatchLength.&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="5188696" author="JIRAUSER1259527" created="Sat, 11 Feb 2023 04:46:04 +0000"  >&lt;p&gt;Hey &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=luke.mauldin%40l3harris.com&quot; class=&quot;user-hover&quot; rel=&quot;luke.mauldin@l3harris.com&quot;&gt;luke.mauldin@l3harris.com&lt;/a&gt; thanks for the ticket and the update with lots of detail! I&apos;m going to start looking into this early next week. I have a few questions:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;Did you start noticing the high CPU usage when you upgraded the Go driver? If so, what is the latest version of the Go driver that doesn&apos;t cause the high CPU usage in your application?&lt;/li&gt;
	&lt;li&gt;Can you share snippets of the code that reads the tailable cursor (i.e. calls &lt;tt&gt;Next&lt;/tt&gt; or &lt;tt&gt;TryNext&lt;/tt&gt;) before and after the code update you mentioned?&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="5173857" author="luke.mauldin@l3harris.com" created="Tue, 7 Feb 2023 14:08:43 +0000"  >&lt;p&gt;Documenting additional information. &#160;The original code using tailable cursors was written when Mongo driver v1.0.0 was released and had not been updated since that time. &#160;It was using cursor.Next(). &#160;I updated the code based on the example ExampleCursor_RemainingBatchLength which uses cursor.TryNext() and cursor.RemainingBatchLength(). &#160;&lt;/p&gt;

&lt;p&gt;Using the updated code seems to have resolved the performance problem. &#160;&lt;/p&gt;

&lt;p&gt;Should the documentation be updated to strongly recommend the use of cursor.TryNext() when using a TailableAwaitable cursor?&lt;/p&gt;

&lt;p&gt;I am also including a pprof screenshot that I captured during a time when the application was using 100% of one CPU and in an almost locked state. &lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/430432/430432_mongo_driver_pprof.png&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                            <comment id="5173807" author="luke.mauldin@l3harris.com" created="Tue, 7 Feb 2023 14:00:45 +0000"  >&lt;p&gt;Please update the title to:&lt;/p&gt;
&lt;h1&gt;&lt;a name=&quot;Highgoroutinecountinconnectionread%2Fwrite&quot;&gt;&lt;/a&gt;High goroutine count in connection read/write&lt;/h1&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="430432" name="mongo_driver_pprof.png" size="126812" author="luke.mauldin@l3harris.com" created="Tue, 7 Feb 2023 14:08:32 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_14266" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Documentation Changes Summary</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;1.  What would you like to communicate to the user about this feature?&lt;br/&gt;
2.  Would you like the user to see examples of the syntax and/or executable code and its output?&lt;br/&gt;
3.  Which versions of the driver/connector does this apply to?&lt;/p&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1cogg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            </customfields>
    </item>
</channel>
</rss>