<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:23:40 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DRIVERS-1589] Allow changing BatchSize for getMores; needed when BatchSize(0) uses default size</title>
                <link>https://jira.mongodb.org/browse/DRIVERS-1589</link>
                <project id="10980" key="DRIVERS">Drivers</project>
                    <description>&lt;h3&gt;&lt;a name=&quot;ProblemStatement%2FRationale&quot;&gt;&lt;/a&gt;&lt;b&gt;Problem Statement/Rationale&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;Currently, when the BatchSize(0) option is used to request an immediate cursor (and no docs), the default BatchSize goes into effect for subsequent getMores and there&apos;s apparently no way to specify a custom BatchSize in this scenario.&#160;&lt;/p&gt;

&lt;p&gt;(This is because BatchSize is being overloaded to both set the BatchSize and to request an immediate cursor on the first aggregation.)&lt;/p&gt;

&lt;p&gt;This is problematic currently in ADL when we need to use BatchSize(0) but the number of docs returned in subsequent batches is so large that our processing exceeds servers cursor expiry threshold.&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Proposed Change/Action&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;Support some mechanism to set BatchSize for subsequent getMores; or barring that, provide a new mechanism to get the first cursor without any docs and leave BatchSize for specifying all batch sizes.&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;ExpectedImpact&quot;&gt;&lt;/a&gt;&lt;b&gt;Expected Impact&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;&lt;font color=&quot;#505f79&quot;&gt;It will provide nice flexibility and allow them to more easily do things that are difficult now.&lt;/font&gt;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;UX Notes (Acceptance Criteria)&lt;/b&gt;&lt;br/&gt;
Have the&#160;functionality of&#160;BatchSize(0) for&#160;&#160;requesting an immediate cursor and the ability to specify a custom BatchSize for subsequent getMores/Nexts.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Open Questions&lt;/b&gt;&lt;/p&gt;

&lt;p&gt;&lt;font color=&quot;#505f79&quot;&gt;&lt;em&gt;If there are alternative ways of improving the situation, include that information here.&lt;/em&gt;&lt;/font&gt;&lt;/p&gt;
&lt;h3&gt;&lt;a name=&quot;AdditionalNotes&quot;&gt;&lt;/a&gt;&lt;b&gt;Additional Notes&lt;/b&gt;&lt;/h3&gt;

&lt;p&gt;&lt;font color=&quot;#505f79&quot;&gt;&lt;em&gt;Any additional information that may be useful to include.&lt;/em&gt;&lt;/font&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="1636324">DRIVERS-1589</key>
            <summary>Allow changing BatchSize for getMores; needed when BatchSize(0) uses default size</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="lorenz.huelsbergen@mongodb.com">Lorenz Huelsbergen</reporter>
                        <labels>
                    </labels>
                <created>Mon, 1 Mar 2021 21:50:54 +0000</created>
                <updated>Fri, 24 Feb 2023 18:41:08 +0000</updated>
                                                                                    <votes>0</votes>
                                    <watches>9</watches>
                                                                                                                <comments>
                            <comment id="3664880" author="JIRAUSER1258051" created="Mon, 15 Mar 2021 17:32:48 +0000"  >&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;(fyi- i was not receiving updates to the ticket for a bit it seems, but it looks like proper permissions have now been restored.)&lt;/p&gt;

&lt;p&gt;as far as more context:&#160; ADL needs to two things ideally when running an aggregation from a driver:&#160; 1) immediate access to the cursor w/o any docs; and 2) the ability to set a batch size other than the default.&lt;/p&gt;

&lt;p&gt;point 1 is necessary given ADL&apos;s model where an ADL client query may be interrupted/canceled at any point by the client.&#160; we do this on the Atlas side by killing a cursor.&#160; therefore, we need the cursor immediately and cannot wait for the first batch of docs to return.&lt;/p&gt;

&lt;p&gt;point 2 is necessary because sometimes the batch size is too large and ADL receives too many docs which cannot be processed (consider: many small writes to S3) before the Atlas cursor expires (10m i believe).&#160; if we can set a batch size we can circumvent this.&#160; it would be ideal to be able to set the batch size dynamically as the query progresses; however, setting it once statically should be sufficient for us to avoid this scenario.&lt;/p&gt;

&lt;p&gt;currently, points 1 &amp;amp; 2 are mutually exclusive since point 1 is conveyed with a batch size of zero, but this disallows setting a batch size smaller than the default.&#160; conversely, setting a batch size smaller than the default (but not zero) precludes getting a cursor immediately.&lt;/p&gt;

&lt;p&gt;there are many ways to solve this on the driver side.&#160; for example, provide a separate aggregation option to get the cursor w/o any docs and keep BatchSize for setting batch sizes other than the default.&lt;/p&gt;

&lt;p&gt;glad to provide more info where needed; please let me know.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="3664789" author="alexander.golin" created="Mon, 15 Mar 2021 17:03:21 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=lorenz.huelsbergen&quot; class=&quot;user-hover&quot; rel=&quot;lorenz.huelsbergen&quot;&gt;lorenz.huelsbergen&lt;/a&gt;, driver leads are discussing in triage and are wondering if you could update this ticket with more detail about the motivation for this request. Thank you! &lt;/p&gt;</comment>
                            <comment id="3655690" author="jeff.yemin" created="Tue, 9 Mar 2021 21:47:48 +0000"  >&lt;p&gt;The request is specific to ADL&apos;s use of the Go driver to communicate with backing Atlas clusters.  ADL does not need this capability in all drivers (Note that the issue was originally opened as &lt;a href=&quot;https://jira.mongodb.org/browse/GODRIVER-1900&quot; title=&quot;Add SetBatchSize method to driver.BatchCursor&quot; class=&quot;issue-link&quot; data-issue-key=&quot;GODRIVER-1900&quot;&gt;&lt;del&gt;GODRIVER-1900&lt;/del&gt;&lt;/a&gt; and subsequently cloned here.  I didn&apos;t realize that when we first triaged it).  While it may make sense to expose this capability in all drivers, I don&apos;t think that the issue has not been raised often enough in the general community to justify the effort.&lt;/p&gt;

&lt;p&gt;Furthermore, there is the possibility to expose this feature through the existing unstable BatchCursor API in the Go driver.  If this is time-sensitive for ADL, I suggest that we explore that route.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10720">
                    <name>Cloners</name>
                                            <outwardlinks description="clones">
                                        <issuelink>
            <issuekey id="1633687">GODRIVER-1900</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                        <issuelink>
            <issuekey id="1614269">PYTHON-2561</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1340603">GODRIVER-1605</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="1614408">PYTHON-2563</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1932311">DRIVERS-1992</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                                                                                                                                                                                                                                                                            <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyioq7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        </customfields>
    </item>
</channel>
</rss>