<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:30:23 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-53257] Determine whether we need const CollectionShardingState/DatabaseShardingState instances for Lock-Free Reads</title>
                <link>https://jira.mongodb.org/browse/SERVER-53257</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Currently we&apos;re adding shared_ptr getters to the interfaces in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-51319&quot; title=&quot;Check DatabaseShardingState::checkDbVersion after acquiring a storage snapshot for AutoGetCollection*LFR&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-51319&quot;&gt;&lt;del&gt;SERVER-51319&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Ideally, CSS::getShared, etc., would return const pointers, but would require further interface changes to accept const pointers. Also, without const pointers, or more accurately without copy-on-write, the metadata instance fetched can be modified during a read. (internally, Collection/DatabaseShardingState have concurrency control via mutex, but state still changes??)&lt;/p&gt;

&lt;p&gt;Maybe this is safe, but need to double check, particularly with things like the resharding project coming up?&lt;/p&gt;</description>
                <environment></environment>
        <key id="1561202">SERVER-53257</key>
            <summary>Determine whether we need const CollectionShardingState/DatabaseShardingState instances for Lock-Free Reads</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="dianna.hohensee@mongodb.com">Dianna Hohensee</assignee>
                                    <reporter username="dianna.hohensee@mongodb.com">Dianna Hohensee</reporter>
                        <labels>
                    </labels>
                <created>Mon, 7 Dec 2020 16:44:14 +0000</created>
                <updated>Mon, 8 Feb 2021 20:15:16 +0000</updated>
                            <resolved>Mon, 8 Feb 2021 20:15:16 +0000</resolved>
                                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="3605346" author="dianna.hohensee" created="Mon, 8 Feb 2021 20:15:16 +0000"  >&lt;p&gt;I have filed several SERVER tickets based on the above investigation and thoughts. Closing this ticket as resolved.&lt;/p&gt;</comment>
                            <comment id="3600554" author="dianna.hohensee" created="Thu, 4 Feb 2021 23:02:09 +0000"  >&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;&lt;tt&gt;DatabaseShardingState&lt;/tt&gt; and &lt;tt&gt;CollectionShardingState&lt;/tt&gt;&#160;use &lt;tt&gt;DSSLock&lt;/tt&gt;&#160;and &lt;tt&gt;CSSLock&lt;/tt&gt;&#160;mutex RAII types that can only be accessed AFTER the &lt;tt&gt;DSS&lt;/tt&gt;&#160;or &lt;tt&gt;CSS&lt;/tt&gt;&#160;is fetched. A collection or database level-lock currently protects the &lt;tt&gt;DSS::get()&lt;/tt&gt;&#160;and &lt;tt&gt;CSS::get()&lt;/tt&gt;&#160;methods. We will not have coll and db locks in not too distant future, so these expectations must change. Either &lt;tt&gt;DSS&lt;/tt&gt;&#160;and &lt;tt&gt;CSS&lt;/tt&gt;&#160;must always be safe to access without lock or mutex &#8211; the current implementation is, but not officially &#8211;, or the &lt;tt&gt;DSSLock&lt;/tt&gt;&#160;and &lt;tt&gt;CSSLock&lt;/tt&gt;&#160;concepts need to work somehow before fetching the &lt;tt&gt;DSS&lt;/tt&gt;&#160;or &lt;tt&gt;CSS&lt;/tt&gt;. (currently the &lt;tt&gt;DSS/CSSLock&lt;/tt&gt;&#160;types lock the mutex IN the &lt;tt&gt;DSS/CSS&lt;/tt&gt;).
	&lt;ul&gt;
		&lt;li&gt;FYI, the Execution team has intentions of removing the &lt;tt&gt;Database&lt;/tt&gt;&#160;class eventually. Today, it mostly behaves as a wrapper around operations applied to all &lt;tt&gt;Collection}}s in the {{CollectionCatalog&lt;/tt&gt;&#160;with the same db namespace &#8211; lots of for loops. It still contains a small amount of in-memory state that needs to be moved elsewhere before we can eliminate &amp;#8211; or rework as a utility library &amp;#8211; the class entirely.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The DatabaseShardingState and CollectionShardingState classes are considered global and unversioned state, I believe, merely accessors for various sharding state. Therefore, it would best to be able to safely fetch versioned state THROUGH the DSS and CSS objects, to then stash for a lock-free read operation to use until finished.
	&lt;ul&gt;
		&lt;li&gt;An alternative would be to make these classes copy-on-write, so a lock-free read can grab a versioned instance of the &lt;tt&gt;CSS&lt;/tt&gt;&#160;or &lt;tt&gt;DSS&lt;/tt&gt;, but that doesn&#8217;t seem desirable from past conversations.&lt;/li&gt;
		&lt;li&gt;As a note, the execution layer &lt;tt&gt;Collection&lt;/tt&gt;&#160;class is copy-on-write and can have several &lt;tt&gt;Collection&lt;/tt&gt;&#160;instances for a single collection. Furthermore, a &lt;tt&gt;Collection&lt;/tt&gt;&#160;instance can have versioned decorations OR unversioned shared decorations via decorating the &lt;tt&gt;ShardCollectionDecorations&lt;/tt&gt;&#160;that exists as shared state across all &lt;tt&gt;Collection&lt;/tt&gt;&#160;instances for a single collection.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The shard filtering/routing information should be moved from the query layer to the &lt;tt&gt;AutoGetCollection*/CollectionPtr&lt;/tt&gt;&#160;interfaces
	&lt;ul&gt;
		&lt;li&gt;With this change, we should reconsider whether the &lt;tt&gt;AutoGetCollection*&lt;/tt&gt;&#160;classes still need to explicitly check the shardVersion, as fetching the filtering metadata does the check implicitly.&lt;/li&gt;
		&lt;li&gt;&lt;tt&gt;ScopedCollectionDescription&lt;/tt&gt;, or whatever class is the filtering metadata, must not have any other sharding linking dependencies, and should be documented strongly to keep it that way. We do NOT want to link sharding into the execution catalog / code layer.&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
	&lt;li&gt;The dbVersion check should be moved out of the &lt;tt&gt;AutoGetDb&lt;/tt&gt;&#160;and &lt;tt&gt;AutoGetCollectionForReadLockFree&lt;/tt&gt;&#160;helpers and instead explicitly called in operations where needed. Execution plans to eliminate database level locking in the near future. &lt;b&gt;(this would likely be Execution work)&lt;/b&gt;&lt;/li&gt;
	&lt;li&gt;Query has a use case for doing an aggregation on a view namespace wherein they must check whether the underlying collection is sharded or not &#8212; an optimization to reduce round trips between mongos and shard. Aggregation is running lock-free, so I recommended using &lt;tt&gt;CSS::getSharedForLockFreeReads()&lt;/tt&gt;&#160;that bypasses the collection lock requirements for lock-free read operations. It seems likely that we will have an increasing need to access the &lt;tt&gt;CSS&lt;/tt&gt;&#160;without collection locks, especially down the line when Execution runs projects to remove more locks.&lt;/li&gt;
	&lt;li&gt;The &lt;tt&gt;CSS::getSharedForLockFreeReads()&lt;/tt&gt;&#160;should return a const object. This requires wider changes to allow the &lt;tt&gt;CSSLock&lt;/tt&gt;&#160;related code, and maybe some other places, to operate on a const &lt;tt&gt;CSS&lt;/tt&gt;&#160;object. This would prevent ostensible readers from running &lt;tt&gt;CSS&lt;/tt&gt;&#160;functions that may modify the &lt;tt&gt;CSS&lt;/tt&gt;&#160;or other sub-state, which is not currently protected against.
	&lt;ul&gt;
		&lt;li&gt;Sharding may want to consider changes to the &lt;tt&gt;CSS&lt;/tt&gt;&#160;and/or &lt;tt&gt;DSS&lt;/tt&gt;&#160;classes such that there can be const readers and writers such that readers will not be allowed/able to do something they should not by accident&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;@ &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; these are the changes that I&apos;ve thought of to make the Execution-Sharding layers smoother in a lock-free and forward thinking world.&#160;They are probably a good discussion starting point. at least. I tried to limit the suggestions to those at immediate hand, and to provide some larger context of which to be aware. CC&#160;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                            <outwardlinks description="duplicates">
                                        <issuelink>
            <issuekey id="1570425">SERVER-53392</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="1616472">SERVER-54401</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1616491">SERVER-54402</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1616501">SERVER-54403</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1616521">SERVER-54404</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="1499218">SERVER-51319</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>dianna.hohensee@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hykmjj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hye4rr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4330">Execution Team 2020-12-28</customfieldvalue>
    <customfieldvalue id="4365">Execution Team 2021-01-11</customfieldvalue>
    <customfieldvalue id="4366">Execution Team 2021-01-25</customfieldvalue>
    <customfieldvalue id="4552">Execution Team 2021-02-08</customfieldvalue>
    <customfieldvalue id="4553">Execution Team 2021-02-22</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyk8sv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>