<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:51:48 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-83303] Attempt to read whole message in one go in sourceMessage</title>
                <link>https://jira.mongodb.org/browse/SERVER-83303</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-81784&quot; title=&quot;Pass MSG_WAITALL to send/recv when doing sync networking&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-81784&quot;&gt;&lt;del&gt;SERVER-81784&lt;/del&gt;&lt;/a&gt;, Mathias recommended that we optimize the sync networking path on linux by passing MSG_WAITALL at all times for sinkMessage. He also recommended this improvement for sourceMessage, which I split into this ticket because of its additional complexity:&lt;/p&gt;

&lt;p&gt;While we are at it, we should make the&#160;&lt;tt&gt;sourceMessage&lt;/tt&gt;&#160;a bit more optimal for small messages as well. Currently we do a recv of 16 bytes to read the header, then do another recv to read the rest of the message. Instead we should allocate a buffer on the stack (maybe 1, 4, or 16KB?) and do a recv into that, only looping until we have the size (in general we won&apos;t loop at all). If we got lucky and got a full message on our first try, we can just copy that into a&#160;&lt;tt&gt;Message&lt;/tt&gt;&#160;and move on without doing a second syscall. If we didn&apos;t get a full message then we should to a&#160;&lt;tt&gt;recv(MSG_WAITALL)&lt;/tt&gt;&#160;for the remainder.&lt;/p&gt;

&lt;p&gt;As noted in a discussion on slack : &quot;I think this is a &apos;strict win&apos; for us - we either do a &apos;big read&apos; and avoid doing the second syscall, or we have the &apos;normal behavior&apos; where we read the header (and then some) in one syscall, and then do a second to get the rest.&quot;&lt;/p&gt;

&lt;p&gt;After discussing with the team, we noted that for exhaust commands, we may run into a situation where this initial read reads in multiple messages at once. In this case, we would need to parse the contents of the buffer to return one message from sourceMessage, and stash the remaining messages to be returned on future calls to sourceMessage.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2501283">SERVER-83303</key>
            <summary>Attempt to read whole message in one go in sourceMessage</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10018" iconUrl="https://jira.mongodb.org/images/icons/statuses/visible.png" description="">In Code Review</status>
                    <statusCategory id="4" key="indeterminate" colorName="inprogress"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="erin.mcnulty@mongodb.com">Erin McNulty</assignee>
                                    <reporter username="erin.mcnulty@mongodb.com">Erin McNulty</reporter>
                        <labels>
                            <label>perf-8.0</label>
                            <label>perf-tiger</label>
                            <label>perf-tiger-handoff</label>
                    </labels>
                <created>Wed, 15 Nov 2023 21:30:22 +0000</created>
                <updated>Mon, 5 Feb 2024 14:47:42 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="5908288" author="redbeard0531" created="Tue, 28 Nov 2023 09:16:28 +0000"  >&lt;p&gt;Re updated psuedo-code, you also need to handle the case where you don&apos;t have enough of the message ready yet to know the full message size. Probably can just add a loop around the initial recv until you have enough bytes. Just make sure to update the ptr and length.&lt;/p&gt;</comment>
                            <comment id="5907202" author="JIRAUSER1260475" created="Mon, 27 Nov 2023 21:12:45 +0000"  >&lt;p&gt;Essentially what we need to write is something like this in sourceMessage:&lt;br/&gt;
&lt;tt&gt;if sync:&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;&#160; recv into buffer of ~1024 bytes&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;&#160; if messageSize &amp;lt; 1024:&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;&#160; &#160; check if there is more than one message in the buffer-- if there is and we have one or more additional complete messages, stash those messages because they are part of an exhaust query or a fire and forget command&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;&#160; else:&lt;/tt&gt;&lt;br/&gt;
&lt;tt&gt;&#160; &#160; recv(...., MSG_WAITALL) // get the rest of the message&lt;/tt&gt;&lt;br/&gt;
&#160;&lt;br/&gt;
along with creating a mechanism for stashing the overflow messages and deciding on what behavior we should use if we have read, for example, 1.5 messages (although the simplest behavior is probably to just do a recv(MSG_WAITALL) on the last message).&#160;&lt;/p&gt;</comment>
                            <comment id="5884358" author="redbeard0531" created="Thu, 16 Nov 2023 08:38:51 +0000"  >&lt;p&gt;Note that it isn&apos;t just for exhaust replies but also fire-and-forget requests. At the OP_MSG layer both are indicated with the kMoreToCome bit being set, and I think both go through sourceMessage(), but we should ensure that if there is any asymmetry in how they are handled, both cases work correctly.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25132"><![CDATA[Service Arch]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 16 Nov 2023 08:38:51 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>phoebe.du@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>erin.mcnulty@mongodb.com</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i30o53:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i2ihkk:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="8038">Service Arch 2024-01-08</customfieldvalue>
    <customfieldvalue id="8039">Service Arch 2024-01-22</customfieldvalue>
    <customfieldvalue id="8040">Service Arch 2024-02-05</customfieldvalue>
    <customfieldvalue id="8041">Service Arch 2024-02-19</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i30aaf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>