<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:24:57 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-73536] Account for the size of the outgoing request in bulkWrite sub-batching logic</title>
                <link>https://jira.mongodb.org/browse/SERVER-73536</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When constructing the sub-batch bulkWrite command to send to shards, we should accounts for the size of the outgoing request, similar to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/9bdd17de7c41ba7cdf7ec330b911d1a141e46d94/src/mongo/s/write_ops/batch_write_op.cpp#L377-L402&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this&lt;/a&gt;. &lt;/p&gt;</description>
                <environment></environment>
        <key id="2250408">SERVER-73536</key>
            <summary>Account for the size of the outgoing request in bulkWrite sub-batching logic</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="kaitlin.mahar@mongodb.com">Kaitlin Mahar</assignee>
                                    <reporter username="lingzhi.deng@mongodb.com">Lingzhi Deng</reporter>
                        <labels>
                            <label>milestone-2</label>
                    </labels>
                <created>Thu, 2 Feb 2023 04:56:01 +0000</created>
                <updated>Sun, 29 Oct 2023 21:27:01 +0000</updated>
                            <resolved>Sun, 25 Jun 2023 15:11:44 +0000</resolved>
                                                    <fixVersion>7.1.0-rc0</fixVersion>
                                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="5522572" author="xgen-internal-githook" created="Fri, 23 Jun 2023 23:20:56 +0000"  >&lt;p&gt;Author: &lt;/p&gt;
{&apos;name&apos;: &apos;Kaitlin Mahar&apos;, &apos;email&apos;: &apos;kaitlin.mahar@mongodb.com&apos;, &apos;username&apos;: &apos;kmahar&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-73536&quot; title=&quot;Account for the size of the outgoing request in bulkWrite sub-batching logic&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-73536&quot;&gt;&lt;del&gt;SERVER-73536&lt;/del&gt;&lt;/a&gt; Account for the size of the outgoing request in bulkWrite sub-batching logic&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/d3c6f9a58fc73147ed83dbec0276d5f66f3ffeaa&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/d3c6f9a58fc73147ed83dbec0276d5f66f3ffeaa&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="5479860" author="kaitlin.mahar" created="Wed, 7 Jun 2023 00:34:07 +0000"  >&lt;p&gt;On further inspection, we currently always include all the namespaces from the incoming client request in the sub-batches, and don&apos;t factor in whether any writes in the sub-batch target each namespace. So instead we can just calculate up front the size of the nsInfo and factor that into our command size estimation.&lt;/p&gt;

&lt;p&gt;Per conversation with the QE team, we should also factor in top-level fields in the command which could be quite large, e.g. &lt;tt&gt;let&lt;/tt&gt; (see &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-74806&quot; title=&quot;Write size estimation logic does not account for runtime/let constants &quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-74806&quot;&gt;&lt;del&gt;SERVER-74806&lt;/del&gt;&lt;/a&gt;). (Relatedly, we do not factor in those top-level fields on mongos for existing batched writes; I filed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-77653&quot; title=&quot;Batch write size estimation logic on mongos doesn&amp;#39;t account for top-level command fields&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-77653&quot;&gt;SERVER-77653&lt;/a&gt; to cover doing that.)&lt;/p&gt;

&lt;p&gt;We &lt;em&gt;should&lt;/em&gt; also factor in the size of command metadata e.g. $audit but &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-53387&quot; title=&quot;Large internal metadata can trigger BSONObjectTooLarge for commands under the BSON size limit&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-53387&quot;&gt;SERVER-53387&lt;/a&gt; already covers fixing that problem for all commands so we will not attempt to tackle it here.&lt;/p&gt;</comment>
                            <comment id="5379240" author="kaitlin.mahar" created="Wed, 26 Apr 2023 21:50:17 +0000"  >&lt;p&gt;Wanted to note while I&apos;ve been looking at this code that making &lt;tt&gt;getWriteSizeBytes()&lt;/tt&gt; work for bulkWrite may be a little tricky. Right now that method only produces estimates that are valid for how inserts, updates, and deletes are represented in the insert/update/delete commands respectively, and the method only has to think about the size of the op itself. &lt;br/&gt;
But since bulkWrite allows multiple namespaces, we should probably consider for each operation whether or not the batch already contains an operation with the same namespace. If it does, we only have to think about the size of the op itself; but if it doesn&apos;t, then the command size would need to grow by both the size of the op &lt;em&gt;and&lt;/em&gt; the size of its namespace.&lt;br/&gt;
We could be conservative and just always assume the namespace isn&apos;t already present, but I think for workloads where many ops go to the same namespace we could end up significantly underfilling batches with that approach. &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="2231752">SERVER-72787</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2355026">SERVER-77653</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 26 Apr 2023 21:50:17 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        32 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-72787'>SERVER-72787</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1452</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            32 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>kaitlin.mahar@mongodb.com</customfieldvalue>
            <customfieldvalue>lingzhi.deng@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1tkhb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1c1s0:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_22250" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Special Downgrade Instructions Required</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="23343"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="6862">Repl 2023-02-20</customfieldvalue>
    <customfieldvalue id="6864">Repl 2023-03-20</customfieldvalue>
    <customfieldvalue id="7171">Repl 2023-05-15</customfieldvalue>
    <customfieldvalue id="7172">Repl 2023-05-29</customfieldvalue>
    <customfieldvalue id="7508">Repl 2023-06-12</customfieldvalue>
    <customfieldvalue id="7509">Repl 2023-06-26</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1t6mn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>