<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:40:52 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-57058] Revisiting the ShardSeverCatalogCacheLoader to properly handle creation-drop-creation scenarios</title>
                <link>https://jira.mongodb.org/browse/SERVER-57058</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;The goal of this ticket is to verify that the &lt;tt&gt;ShardServerCatalogCacheLoader&lt;/tt&gt; properly works when the information that we get &lt;a href=&quot;#L680-L681&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;from the config server&lt;/a&gt; has a different epoch than the information that we have locally.&lt;/p&gt;

&lt;p&gt;Note that every time the &lt;tt&gt;SSCCL&lt;/tt&gt; asks for information to the config server, &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L745&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;it creates a task that contains this information&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L742-L745&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;it is appended to a list of pending tasks&lt;/a&gt;. This list of tasks is used for two things:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;In order to fulfill the refresh that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/s/catalog_cache.cpp#L607&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;comes from the catalog cache&lt;/a&gt;, the &lt;tt&gt;SSCCL&lt;/tt&gt; has to compute the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L763-L764&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;results of the refresh&lt;/a&gt;. This usually &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L829-L931&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;combines some persisted metadata with the metadata of the tasks that haven&apos;t been persisted yet&lt;/a&gt;.&lt;/li&gt;
	&lt;li&gt;Using an external thread, the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L1024-L1097&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;changes associated to each task are locally persisted on config.cache.collections and config.cache.chunks.*&lt;/a&gt;.&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;So far we know that the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/d1982e704de77f3abe0a71e770b983a6e79f6ea1/src/mongo/db/s/shard_server_catalog_cache_loader.cpp#L1362-L1383&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;following two invariants are incorrect under this scenario&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;I see two ways of supporting this scenario:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;When we detect an epoch mismatch, instead of creating one task we create two: the first one represents the drop of the old collection so and the new one is just the re-creation. From a CollectionVersion point of view, it will be: CV_old -&amp;gt; UNSHARDED -&amp;gt; CV_new, so we won&apos;t hit the invariants.&lt;/li&gt;
	&lt;li&gt;Having just one task and fixing the invariant an other potential places. Several parts of the code are already prepared to deal with this creation-drop-creation scenario just using one task (I don&apos;t want to add more pointers here but I can walk you through). I wonder how much work we would need to do.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Finally, lucky for us we can unittest the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/shard_server_catalog_cache_loader_test.cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;SSCCL&lt;/tt&gt;&lt;/a&gt;, so I would start creating a unittest for this scenario. We may need a way to start/stop the external thread that consumes tasks.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1730945">SERVER-57058</key>
            <summary>Revisiting the ShardSeverCatalogCacheLoader to properly handle creation-drop-creation scenarios</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="sergi.mateo-bellido@mongodb.com">Sergi Mateo Bellido</assignee>
                                    <reporter username="sergi.mateo-bellido@mongodb.com">Sergi Mateo Bellido</reporter>
                        <labels>
                            <label>PM-1965-Cleanup</label>
                    </labels>
                <created>Wed, 19 May 2021 11:56:52 +0000</created>
                <updated>Wed, 9 Jun 2021 10:06:25 +0000</updated>
                            <resolved>Wed, 9 Jun 2021 09:52:58 +0000</resolved>
                                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="3868051" author="JIRAUSER1256927" created="Wed, 9 Jun 2021 09:52:58 +0000"  >&lt;p&gt;Will be done as part of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34632&quot; title=&quot;config.chunks change to config.cache.chunks creates a collection long name after upgrade&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34632&quot;&gt;SERVER-34632&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 35 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1965</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>sergi.mateo-bellido@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 35 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>sergi.mateo-bellido@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzdcon:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyy2mn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4840">Sharding EMEA 2021-05-31</customfieldvalue>
    <customfieldvalue id="4978">Sharding EMEA 2021-06-14</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hzcyxr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>