<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:10:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-68341] Implement enable/disable command for mongoQ in serverless</title>
                <link>https://jira.mongodb.org/browse/SERVER-68341</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;PM-2334 implemented enable/disable change streams command for the replica sets. The corresponding ticket was: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-66631&quot; title=&quot;Implement enable and disable command for the change collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-66631&quot;&gt;&lt;del&gt;SERVER-66631&lt;/del&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This work for the sharded cluster was punted because there are subtle differences in how mongoS and mongoQ behave. For example, in mongoQ, the above command should dispatch the request to only a subset of shards where the corresponding tenant resides as opposed to mongoS where the request gets sent to all the shards. Since, there can be up to 10k shards (nodes) in the serverless, fanning out requests blindly to all the nodes is not a scalable solution. This ticket is about extending the work done in the &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-66631&quot; title=&quot;Implement enable and disable command for the change collection&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-66631&quot;&gt;&lt;del&gt;SERVER-66631&lt;/del&gt;&lt;/a&gt; to run in the mongoQ.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Since we will be building this enable/disable command using the POS service, we don&apos;t have atomicity when creating change collections on shards, ie. a change collection at shard1 might be made a timestamp t1 and on shard2 at timestamp t2. Whilst in general, this is not a problem for the change stream, it could be a problem for the following rare occurring scenario:&lt;/p&gt;

&lt;p&gt;Consider this:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;shard1 change collection creating ts is t1&lt;/li&gt;
	&lt;li&gt;shard2 change collection creating rs is t2&lt;/li&gt;
	&lt;li&gt;&#160;t1 &amp;lt; t2&lt;/li&gt;
	&lt;li&gt;other than the change collection seed-entry (own create-collection oplog entry), no other entries were written to the change collection, ie. each of the change collections has just one entry, ie the seed-entry.&lt;/li&gt;
	&lt;li&gt;the client immediately opened the change stream cursor and the resume token corresponds to t1.&lt;/li&gt;
	&lt;li&gt;a get-next on shard2 will result in a ChangeStreamHistoryLost error, as the time t2 &amp;gt; provided time t1.&lt;/li&gt;
	&lt;li&gt;this case is similar to the addShard case with the change collection.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Above is not a problem for the oplog because the oplog gets created at a very early stage of the node-bring up, as such the change stream resume token is always greater than the first entry of all the oplog in all the nodes.&#160;&lt;/p&gt;

&lt;p&gt;To mitigate above issue, we could potentially enable a change stream in the mongoQ in 2 stages -&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;the first stage will be to create a change collection in required shards.&lt;/li&gt;
	&lt;li&gt;And the second stage will be to add a dummy oplog entry to the tenant&apos;s change collection in required shards.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;With this, we will ensure that the resume token that the client will operate upon will be greater than the initial timestamp of all the change collections.&lt;/p&gt;

&lt;p&gt;While this is just one solution, we will explore more options when we work on this ticket.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;This work will be under the serverless initiative - &lt;b&gt;Change streams with mongoq&lt;/b&gt;&lt;/p&gt;</description>
                <environment></environment>
        <key id="2099852">SERVER-68341</key>
            <summary>Implement enable/disable command for mongoQ in serverless</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-query-execution">Backlog - Query Execution</assignee>
                                    <reporter username="rishab.joshi@mongodb.com">Rishab Joshi</reporter>
                        <labels>
                            <label>change-streams-w/-mongoq</label>
                            <label>phase3</label>
                            <label>pm3278-m2</label>
                            <label>replace-atlas-proxy-w-mongoq</label>
                    </labels>
                <created>Tue, 26 Jul 2022 21:37:00 +0000</created>
                <updated>Tue, 12 Dec 2023 18:39:48 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="5348850" author="kyle.suarez" created="Fri, 14 Apr 2023 19:27:45 +0000"  >&lt;p&gt;Had a quick discussion with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bernard.gorman%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;bernard.gorman@mongodb.com&quot;&gt;bernard.gorman@mongodb.com&lt;/a&gt; and while it&apos;s not &lt;b&gt;quite&lt;/b&gt; the same motivation as the rest of the tickets in the mongos process interface project, it&apos;s fine to just tack it on to that project. Removing this from director triage.&lt;/p&gt;

&lt;p&gt;Notes from our discussion&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;New implementation would forward the command to shards&lt;/li&gt;
	&lt;li&gt;Main work is probably handling testing and test failures &amp;#8211; there are some differences in our expectations sharded vs unsharded&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="5344944" author="bernard.gorman" created="Thu, 13 Apr 2023 16:17:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=sebastien.mendez%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;sebastien.mendez@mongodb.com&quot;&gt;sebastien.mendez@mongodb.com&lt;/a&gt;, sending this to you for now to find an assignee on your team. Per the conversation above, it seems like we can just broadcast for now and refine the targeting later. We may have to deal with a couple of JS test failures as a result, but we can likely just blocklist those.&lt;/p&gt;</comment>
                            <comment id="5336111" author="janna.golden" created="Mon, 10 Apr 2023 19:16:24 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bernard.gorman%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;bernard.gorman@mongodb.com&quot;&gt;bernard.gorman@mongodb.com&lt;/a&gt; - ah so we will need these commands in the embedded router layer by 8.0 due to the &quot;all clusters sharded in 8.0&quot; plan laid out by the invisible sharding init. In 8.0, the embedded router layer in Serverless will very likely just be the same as the embedded router layer in dedicated clusters - today mongoq technically exists, though it actually just links in almost the entirety of mongos&apos;s link graph (the plan was always for mongoq to be built on top of mongos, and perhaps we&apos;ll actually re-evaluate whether mongos and mongoq should truly be different processes). In any case, in this ticket I think we can add the commands to the &quot;embedded router layer&quot;&apos;s commands library, and fail the command if it&apos;s run without multitenancySupport enbaled - that way we ensure it isn&apos;t run in a dedicated setting.&lt;/p&gt;

&lt;p&gt;We don&apos;t yet have a &quot;tenant shard targeting&quot; protocol, so for now you could probably broadcast the request, including the tenantId sent in the original request. We can always change the targeting behavior once we&apos;ve implemented &quot;tenant shard targeting&quot;.&lt;/p&gt;</comment>
                            <comment id="5333139" author="bernard.gorman" created="Fri, 7 Apr 2023 19:54:45 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=steven.vannelli%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;steven.vannelli@mongodb.com&quot;&gt;steven.vannelli@mongodb.com&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden@mongodb.com&quot;&gt;janna.golden@mongodb.com&lt;/a&gt;: sure, we can try to schedule this ticket - but I thought we&apos;d have to wait until more work has been completed on the &lt;tt&gt;mongoq&lt;/tt&gt; roadmap before it&apos;s feasible to do so. Is my understanding wrong? Is it actually feasible to do this work now?&lt;/p&gt;</comment>
                            <comment id="5332183" author="steven.vannelli" created="Fri, 7 Apr 2023 14:19:00 +0000"  >&lt;p&gt;The Server Serverless team would like to have this ticket completed by 8.0 We have some time but figured we&apos;d put it up for discussion for FY24 Q2 if the team has time. No harm in doing it early &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&#160;&lt;/p&gt;

&lt;p&gt;We might be able to place this in PM-3278 if folks agree with that.&#160;&#160;&lt;/p&gt;

&lt;p&gt;cc &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=janna.golden%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;janna.golden@mongodb.com&quot;&gt;janna.golden@mongodb.com&lt;/a&gt;&#160;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10820">
                    <name>Initiative</name>
                                                                <inwardlinks description="included in Initiative">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2119714">SERVER-69068</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25125"><![CDATA[Query Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 16 Aug 2022 13:07:03 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        42 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/PM-2347'>PM-2347</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-3278</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>mickey.winters@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            42 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-query-execution</customfieldvalue>
            <customfieldvalue>bernard.gorman@mongodb.com</customfieldvalue>
            <customfieldvalue>janna.golden@mongodb.com</customfieldvalue>
            <customfieldvalue>kyle.suarez@mongodb.com</customfieldvalue>
            <customfieldvalue>rishab.joshi@mongodb.com</customfieldvalue>
            <customfieldvalue>steven.vannelli@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i13xhb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i0mqsg:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i13jmn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>