<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:00:37 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-64557] BatchedDeleteStage should consider yielding based on document and time targets</title>
                <link>https://jira.mongodb.org/browse/SERVER-64557</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Documents that are fetched during the BatchedDeletePhase must be refetched if the snapshot changes before the batch is ready for the delete.&#160;&lt;/p&gt;

&lt;p&gt;We need to ensure the YieldPolicy tied to the BatchedDeleteStage limits auto yields that cause the snapshot to be abandoned.&#160;&lt;/p&gt;</description>
                <environment></environment>
        <key id="2002997">SERVER-64557</key>
            <summary>BatchedDeleteStage should consider yielding based on document and time targets</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="josef.ahmad@mongodb.com">Josef Ahmad</assignee>
                                    <reporter username="haley.connelly@mongodb.com">Haley Connelly</reporter>
                        <labels>
                            <label>PM-2227-M3</label>
                    </labels>
                <created>Wed, 16 Mar 2022 14:38:02 +0000</created>
                <updated>Mon, 9 May 2022 14:58:30 +0000</updated>
                            <resolved>Mon, 9 May 2022 14:58:30 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="4535315" author="josef.ahmad" created="Mon, 9 May 2022 14:58:10 +0000"  >&lt;p&gt;With &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-66105&quot; title=&quot;Tune the batch deleter defaults&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-66105&quot;&gt;&lt;del&gt;SERVER-66105&lt;/del&gt;&lt;/a&gt; setting the&#160;&lt;tt&gt;batchedDeletesTargetStagedDocBytes&lt;/tt&gt; default to 2MB, the refetches due to yield do not appear to be significant any longer. The observed rate of refetches continues to be below 1% as before for 1kB-sized documents, and our benchmarks show similarly low refetch rate for larger documents. This leaves the performance unchanged for smaller documents, and brings the batched deleter performance on par with the doc-by-doc deleter for large documents. We&apos;ve also included the &lt;tt&gt;refetchesDueToYield&lt;/tt&gt; diagnostic metric for observability.&lt;/p&gt;

&lt;p&gt;Closing this ticket as Won&apos;t Do, as the batched deleter performance with the current auto-yielding policy appears to be satisfactory.&lt;/p&gt;</comment>
                            <comment id="4525355" author="josef.ahmad" created="Wed, 4 May 2022 12:59:18 +0000"  >&lt;p&gt;We can reduce the refetches considerably by setting a lower default for &lt;tt&gt;batchedDeletesTargetStagedDocBytes&lt;/tt&gt;. By setting this target to 2MB instead of the current 30MB, we get zero refetches with on a mass deletion with 10MB documents, with roughly the same negligible rate of refetches with the 1kB document test. Below are the metrics of the 10MB workload with &lt;tt&gt;batchedDeletesTargetStagedDocBytes&lt;/tt&gt; set to 2MB. Note the throughput speed-up compared to the chart in my previous comment (serverStatus batchedDeletes docs rate).&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/377115/377115_10MB-docs-2MB-stageSizeTarget.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                            <comment id="4479471" author="haley.connelly" created="Wed, 13 Apr 2022 09:48:50 +0000"  >&lt;p&gt;We are still waiting on design decisions for the best course of action here. This ticket is blocked until we come to a consensus.&lt;/p&gt;</comment>
                            <comment id="4450585" author="josef.ahmad" created="Fri, 1 Apr 2022 15:18:04 +0000"  >&lt;p&gt;If it helps to make an informed decision, I&apos;ve run an instrumented version of the server that reports the refetches of staged documents due to yielding.&lt;/p&gt;

&lt;p&gt;On a mass deletion of 1kB sized documents, we refetch less than 1 document every &lt;del&gt;10&lt;/del&gt; 100. (EDIT: fixed typo)&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/369978/369978_1kB.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;

&lt;p&gt;On a mass deletion of 10MB sized documents, we actually end up refetching almost every document. This is probably because the target batch size of 10 documents actually commits 2-3 documents per batch, as we meet the targetBatchTime of 5ms. This means that we&apos;ll likely yield before the next iteration or the following one, so we&apos;re likely to refetch some of the remaining staged documents.&lt;br/&gt;
&lt;span class=&quot;image-wrap&quot; style=&quot;&quot;&gt;&lt;img src=&quot;https://jira.mongodb.org/secure/attachment/369979/369979_10MB+docs.png&quot; width=&quot;100%&quot; style=&quot;border: 0px solid black&quot; /&gt;&lt;/span&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1972345">SERVER-63039</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="369979" name="10MB docs.png" size="37549" author="josef.ahmad@mongodb.com" created="Fri, 1 Apr 2022 15:14:58 +0000"/>
                            <attachment id="377115" name="10MB-docs-2MB-stageSizeTarget.png" size="52043" author="josef.ahmad@mongodb.com" created="Wed, 4 May 2022 12:57:51 +0000"/>
                            <attachment id="369978" name="1kB.png" size="38103" author="josef.ahmad@mongodb.com" created="Fri, 1 Apr 2022 15:14:54 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 1 Apr 2022 15:18:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 39 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-2227</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>josef.ahmad@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 39 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>haley.connelly@mongodb.com</customfieldvalue>
            <customfieldvalue>josef.ahmad@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0nhwv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i06ivw:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="5814">Execution Team 2022-04-18</customfieldvalue>
    <customfieldvalue id="5979">Execution Team 2022-05-02</customfieldvalue>
    <customfieldvalue id="5980">Execution Team 2022-05-16</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i0n427:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>