<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:35:55 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-34212] Config servers should only upconvert writeConcern to majority for requests on internal connections</title>
                <link>https://jira.mongodb.org/browse/SERVER-34212</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In MongoDB version 3.2 when CSRS was introduced, there were cases where commands coming from mongos did not specify write concern. For this purpose, the config server by default &lt;a href=&quot;https://github.com/mongodb/mongo/blob/431ef23420797fb0956fae4e3d5149e2224bbaee/src/mongo/db/write_concern.cpp#L82&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upconverts the writeConcern to majority&lt;/a&gt;. With the current codebase, this upconversion is no longer necessary because all mongos and shard commands against the config server should be including a write concern.&lt;/p&gt;

&lt;p&gt;This ticket is to add testing that &lt;tt&gt;writeConcern:majority&lt;/tt&gt; is always included with writes against the config server and to remove the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/431ef23420797fb0956fae4e3d5149e2224bbaee/src/mongo/db/write_concern.cpp#L82&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;upconversion code&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="519320">SERVER-34212</key>
            <summary>Config servers should only upconvert writeConcern to majority for requests on internal connections</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="esha.maharishi@mongodb.com">Esha Maharishi</assignee>
                                    <reporter username="spencer@mongodb.com">Spencer Brody</reporter>
                        <labels>
                            <label>neweng</label>
                            <label>sharding-wfbf-day</label>
                    </labels>
                <created>Fri, 30 Mar 2018 15:30:10 +0000</created>
                <updated>Sun, 29 Oct 2023 22:33:11 +0000</updated>
                            <resolved>Mon, 22 Oct 2018 20:35:17 +0000</resolved>
                                                    <fixVersion>4.1.5</fixVersion>
                                    <component>Replication</component>
                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>5</watches>
                                                                                                                <comments>
                            <comment id="2038325" author="xgen-internal-githook" created="Mon, 22 Oct 2018 21:16:16 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Esha Maharishi&apos;, &apos;email&apos;: &apos;esha.maharishi@mongodb.com&apos;, &apos;username&apos;: &apos;EshaMaharishi&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34212&quot; title=&quot;Config servers should only upconvert writeConcern to majority for requests on internal connections&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34212&quot;&gt;&lt;del&gt;SERVER-34212&lt;/del&gt;&lt;/a&gt; Config servers should only upconvert writeConcern to majority for requests on internal connections&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6d9c01b93fb94caf1b6c57bb00bfceb3e329ede9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6d9c01b93fb94caf1b6c57bb00bfceb3e329ede9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2038197" author="esha.maharishi@10gen.com" created="Mon, 22 Oct 2018 20:34:13 +0000"  >&lt;p&gt;Git commit url:&#160;&lt;a href=&quot;https://github.com/mongodb/mongo/commit/6d9c01b93fb94caf1b6c57bb00bfceb3e329ede9&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/6d9c01b93fb94caf1b6c57bb00bfceb3e329ede9&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2025749" author="siyuan.zhou@10gen.com" created="Sat, 6 Oct 2018 02:33:44 +0000"  >&lt;p&gt;Upconversion is always safe, but I think the problem is it has an arbitrary small timeout, 30 seconds, much shorter than the default 10-minute timeout of awaitReplication() in testing. As proposed in BF-8529, a better solution could be only upconverting write concerns or error on unexpected write concerns for &lt;b&gt;internal connections&lt;/b&gt;. From clients or test infrastructure&apos;s point of view, config servers should provide the same API as a normal mongod. Re-opening the ticket.&lt;/p&gt;</comment>
                            <comment id="2012402" author="esha.maharishi@10gen.com" created="Mon, 24 Sep 2018 18:32:13 +0000"  >&lt;p&gt;Spoke to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; to try to (1) clarify and (2) be more succinct:&lt;/p&gt;

&lt;p&gt;In most cases, there is no need for writes to the sharding catalog to be done with writeConcern: majority. All &lt;b&gt;readers&lt;/b&gt; of the catalog read with &lt;b&gt;readConcern:&lt;/b&gt; &lt;b&gt;majority&lt;/b&gt;, so they will only ever see committed data. (It is a separate issue that they may see &lt;em&gt;inconsistent&lt;/em&gt; or &lt;em&gt;partial&lt;/em&gt; data, because catalog updates that involve multiple writes are not always wrapped in transactions. For this reason, readers of the catalog already handle seeing such inconsistent/partial data).&lt;/p&gt;

&lt;p&gt;The places writeConcern: majority &lt;b&gt;is&lt;/b&gt; required are places an action is taken on the config server&apos;s &lt;b&gt;response&lt;/b&gt; (i.e., interpret a success response as meaning the catalog change became majority committed). In contrast, most callers ignore the response and do a refresh of their cache of the catalog to pick up the latest view.&lt;/p&gt;

&lt;p&gt;However, there is no real harm in doing all sharding catalog writes with writeConcern: majority, and there may be edge cases that we haven&apos;t discovered that do depend on it (and that didn&apos;t show up as immediate failures in the Evergreen patch linked above).&lt;/p&gt;

&lt;p&gt;So, we are closing this request as Works as Designed.&lt;/p&gt;</comment>
                            <comment id="1998607" author="esha.maharishi@10gen.com" created="Mon, 10 Sep 2018 20:08:01 +0000"  >&lt;p&gt;The only case where it&apos;s necessary to use majority writeConcern for a write against the config server is if the sending node will then &lt;b&gt;take an action that assumes&lt;/b&gt;&#160;the write will not roll back.&lt;/p&gt;

&lt;p&gt;These are all the collections on the config server:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;config.databases&lt;/li&gt;
	&lt;li&gt;config.collections&lt;/li&gt;
	&lt;li&gt;config.chunks&lt;/li&gt;
	&lt;li&gt;config.shards&lt;/li&gt;
	&lt;li&gt;config.tags&lt;/li&gt;
	&lt;li&gt;config.version&lt;/li&gt;
	&lt;li&gt;config.locks&lt;/li&gt;
	&lt;li&gt;config.lockpings&lt;/li&gt;
	&lt;li&gt;config.settings&lt;/li&gt;
	&lt;li&gt;config.mongos&lt;/li&gt;
	&lt;li&gt;config.changelog&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Here are some actions that might be taken after writing to one of those collections:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;writing user data to the cluster&lt;/li&gt;
	&lt;li&gt;doing another write against the config server&lt;/li&gt;
	&lt;li&gt;sending setShardVersion to one or more shards&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I am most concerned about the first kind of action, writing user data to the cluster. A user &lt;em&gt;should&lt;/em&gt; supply majority writeConcern for metadata commands, but it seems easy for them to forget to do this and end up in a bad state.&lt;/p&gt;

&lt;p&gt;About the second: We already have very weak guarantees around the consistency of the sharding catalog, particularly because the config server doesn&apos;t block nodes from reading the sharding catalog while the config server is writing to it. So, readers of the sharding catalog generally have to handle seeing an inconsistent view and retrying their read.&lt;/p&gt;

&lt;p&gt;About the third: It is harmless to send an extra setShardVersion that ends up being unnecessary because the data it was trying to read rolled back.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;We could, as the ticket suggests, add tests that all commands that write to the sharding catalog assert that the writeConcern is majority. We do in fact already have such a &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/jstests/sharding/configsvr_metadata_commands_require_majority_write_concern.js&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;jstest&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;But it feels highly brittle to me to rely on people remembering to update the jstest whenever they add a metadata command. The sharding team has plans to add metadata commands in the future, such as renameDatabase, renameCollection, and commands to clean up an inconsistent catalog. We have also added metadata commands in the recent past (for shardCollection for the &quot;fast initial split&quot; project in 4.2; for movePrimary for the &quot;database versioning&quot; project in 4.0).&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;I ran a &lt;a href=&quot;https://evergreen.mongodb.com/version/5b96ada3e3c331377e8bf7d3&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;patch&lt;/a&gt;&#160;removing the upconversion, and it came back mostly green (it just caught two places in mapReduce where we internally send &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.1.2/src/mongo/s/commands/cluster_map_reduce_cmd.cpp#L530&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_configsvrDropCollection&lt;/a&gt;&#160;and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r4.1.2/src/mongo/s/commands/cluster_map_reduce_cmd.cpp#L705&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_configsvrShardCollection&lt;/a&gt;&#160;with default writeConcern instead of majority writeConcern).&lt;/p&gt;

&lt;p&gt;However, after speaking to &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=spencer&quot; class=&quot;user-hover&quot; rel=&quot;spencer&quot;&gt;spencer&lt;/a&gt;, I think the safest solution is to continue upconverting writeConcern. Our test infrastructure (and our users) can always avoid the upconversion by supplying their own writeConcern.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="528737">SERVER-34501</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2085061">SERVER-67874</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="371308">SERVER-28641</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 10 Sep 2018 20:08:01 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 16 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 16 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>17.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>esha.maharishi@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htu0o7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hr8omn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2538">Sharding 2018-10-08</customfieldvalue>
    <customfieldvalue id="2540">Sharding 2018-11-05</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|httmwf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>