<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:24:58 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DRIVERS-2194] Clarify the order between server monitor events and connection pool events</title>
                <link>https://jira.mongodb.org/browse/DRIVERS-2194</link>
                <project id="10980" key="DRIVERS">Drivers</project>
                    <description>&lt;p&gt;This &lt;a href=&quot;https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-discovery-and-monitoring-tests.rst#connection-pool-management&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SDAM prose test&lt;/a&gt; implies the following order of events:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;step 2: &lt;tt&gt;ServerHeartbeatSucceededEvent&lt;/tt&gt; happens before &lt;tt&gt;ConnectionPoolReadyEvent&lt;/tt&gt; caused by it&lt;/li&gt;
	&lt;li&gt;steps 4 and 5: &lt;tt&gt;ServerHeartbeatFailedEvent&lt;/tt&gt; happens before &lt;tt&gt;ConnectionPoolReadyEvent&lt;/tt&gt; caused by a successful check that happened after the check that caused &lt;tt&gt;ServerHeartbeatFailedEvent&lt;/tt&gt;.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;However, neither SDAM more CMAP specs require any such ordering. Java driver recently recently started delivering server monitor events asynchronously (in a dedicated thread, together with server and cluster events). This change affected the aforementioned SDAM prose test: it started failing sporadically, and we disabled it. If the order implied by the test is indeed what the specifications require, the order should be explicitly and unambiguously specified either by SDAM or by CMAP specs.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1979074">DRIVERS-2194</key>
            <summary>Clarify the order between server monitor events and connection pool events</summary>
                <type id="14901" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14700&amp;avatarType=issuetype">Spec Change</type>
                                            <priority id="10300" iconUrl="https://jira.mongodb.org/images/icons/priorities/medium.svg">Unknown</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="valentin.kovalenko@mongodb.com">Valentin Kavalenka</reporter>
                        <labels>
                    </labels>
                <created>Mon, 7 Feb 2022 23:39:15 +0000</created>
                <updated>Thu, 31 Mar 2022 14:06:00 +0000</updated>
                                                                <component>SDAM</component>
                                        <votes>1</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="4375853" author="shane.harvey" created="Thu, 24 Feb 2022 22:48:12 +0000"  >&lt;p&gt;I don&apos;t think we should mandate async/sync event emission or ordering just for this spec test. I&apos;m fine with tweaking the test to suggest that the order should be checked only if possible to allow Java to implement it strictly. &lt;/p&gt;</comment>
                            <comment id="4343818" author="JIRAUSER1258163" created="Tue, 8 Feb 2022 22:29:27 +0000"  >&lt;blockquote&gt;&lt;p&gt;it seems like a limitation of the Java driver that events aren&apos;t guaranteed to be emitted in the order that they occur&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The reason this happened is that neither SDAM nor CMAP specs specify any order relations between SDAM and CMAP events. I.e., the recent change in the Java driver did not violate driver specs, yet it made the aforementioned prose test flaky, and we discovered that it fails, and found the reason only after some time since the change was made. To reduce the chances of situations like this one happening, we need to express the order relation requirements in the specs, instead of letting implementors to derive the implied requirements from a single test that fails occasionally if the requirements are not met.&lt;/p&gt;

&lt;p&gt;Are there objections to the idea of explicitly expressing in the driver spec the implied requirements?&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;can you think of a way to rewrite the prose test so that it doesn&apos;t rely on undefined behavior?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;If it is decided that there are no order relations between SDAM and CMAP events, then the test can be rephrased to check the order of SDAM events (&lt;tt&gt;ServerHeartbeatSucceededEvent&lt;/tt&gt; -&amp;gt; &lt;tt&gt;ServerHeartbeatFailedEvent&lt;/tt&gt; -&amp;gt; &lt;tt&gt;ServerHeartbeatSucceededEvent&lt;/tt&gt;) and the order of CMAP events (&lt;tt&gt;ConnectionPoolReadyEvent&lt;/tt&gt; -&amp;gt; &lt;tt&gt;ConnectionPoolClearedEvent&lt;/tt&gt; -&amp;gt; &lt;tt&gt;ConnectionPoolReadyEvent&lt;/tt&gt;) without checking order across the SDAM/CMAP boundary. This is stricter than just checking that events occurred.&lt;/p&gt;</comment>
                            <comment id="4343396" author="patrick.freed" created="Tue, 8 Feb 2022 20:08:44 +0000"  >&lt;p&gt;The test&apos;s goal is to verify the causal relationship between successful heartbeat checks and pools being marked as ready, so if we remove the ordering requirement it loses some of its purpose. I also think that, from a debugging user&apos;s perspective, the distinction between families of events probably isn&apos;t really significant (e.g. CMAP vs SDAM events), but the ordering of them is, especially if there&apos;s a causal relationship between two of them.&lt;/p&gt;

&lt;p&gt;All that being said, it&apos;s not really &lt;em&gt;that&lt;/em&gt; important, so if it would be a lot of work for Java to meet this requirement, it may suffice to just assert that the events were seen. But again, in my opinion, it seems like a limitation of the Java driver that events aren&apos;t guaranteed to be emitted in the order that they occur, and that it may present further challenges testing or debugging other features down the road.&lt;/p&gt;</comment>
                            <comment id="4343268" author="shane.harvey" created="Tue, 8 Feb 2022 19:30:40 +0000"  >&lt;p&gt;Thanks Valentin. My preference would be to avoid mandating an order and instead change or remove the prose test if possible.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=patrick.freed&quot; class=&quot;user-hover&quot; rel=&quot;patrick.freed&quot;&gt;patrick.freed&lt;/a&gt;, can you think of a way to rewrite &lt;a href=&quot;https://github.com/mongodb/specifications/blob/525dae0/source/server-discovery-and-monitoring/server-discovery-and-monitoring-tests.rst#connection-pool-management&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;the prose test&lt;/a&gt; so that it doesn&apos;t rely on undefined behavior?&lt;/p&gt;</comment>
                            <comment id="4343166" author="JIRAUSER1258163" created="Tue, 8 Feb 2022 18:56:11 +0000"  >&lt;p&gt;Currently, my understanding is as follows. The most straightforward way is to start delivering CMAP events in the same thread that delivers SDAM events. However, depending on implementations of user-supplied &lt;a href=&quot;https://mongodb.github.io/mongo-java-driver/4.4/apidocs/mongodb-driver-core/com/mongodb/event/ConnectionPoolListener.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;ConnectionPoolListener&lt;/tt&gt;&lt;/a&gt;s, this may become a performance bottleneck as the rate of &lt;tt&gt;ConnectionCheckedOutEvent&lt;/tt&gt;/&lt;tt&gt;ConnectionCheckedInEvent&lt;/tt&gt; events for the whole &lt;tt&gt;MongoClient&lt;/tt&gt; is not lower than the rate of user operations, which may be quite high. This means that we may decide to deliver a subset of CMAP events in the same thread that delivers SDAM events, e.g., only &lt;tt&gt;ConnectionPoolReadyEvent&lt;/tt&gt; and &lt;tt&gt;ConnectionPoolClearedEvent&lt;/tt&gt;, while delivering the rest of events in the same thread that raised them, i.e., synchronously (this is how all CMAP events are delivered currently). If we choose this path, the subset of CMAP events delivered in the same thread as SDAM events will depend on the order relations between CMAP and SDAM events.&lt;/p&gt;

&lt;p&gt;It is also possible to not specify any order relations between CMAP and SDAM events, but then we need to update the prose test mentioned in the description (this test appears to be the only place that depends on such order relations).&lt;/p&gt;</comment>
                            <comment id="4343085" author="shane.harvey" created="Tue, 8 Feb 2022 18:36:50 +0000"  >&lt;p&gt;If we changed the spec to mandate this order of events, how would Java implement that requirement? The answer could impact our decision.&lt;/p&gt;</comment>
                            <comment id="4342296" author="JIRAUSER1258163" created="Tue, 8 Feb 2022 15:25:07 +0000"  >&lt;blockquote&gt;&lt;p&gt;expose the creation time of each event? Then the events could be sorted to the correct order regardless of async or sync publishing&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Non-logical timestamps cannot be used to impose/derive order between actions under the Java memory model, even if they come from a monotonic clock. The same must be true for all other driver languages/runtimes except maybe (I have no idea) PHP, Node.js, Ruby, Python.&lt;/p&gt;

&lt;p&gt;Anyway, what you are talking, Shane, is an implementation detail, while this ticket is about the specification requirements. For example, SDAM tries to express ordering constraints on its events &lt;a href=&quot;https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-discovery-and-monitoring-monitoring.rst#event-order-and-concurrency&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;, but there is no place that expresses order relations between SDAM and CMAP events.&lt;/p&gt;</comment>
                            <comment id="4341075" author="shane.harvey" created="Tue, 8 Feb 2022 01:00:57 +0000"  >&lt;p&gt;It would surely be unexpected for me to see a ConnectionPoolReadyEvent before a ServerHeartbeatSucceededEvent since the former requires the later. Could the Java driver expose the creation time of each event? Then the events could be sorted to the correct order regardless of async or sync publishing. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1979079">JAVA-4484</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                                                <inwardlinks description="is caused by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1978981">JAVA-4483</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                            <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10951" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Driver Changes</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10748"><![CDATA[Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i02j6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>