<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:13:53 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-27005] Write error revalidate logic needs to wait for lastVisibleOpTime to be committed</title>
                <link>https://jira.mongodb.org/browse/SERVER-27005</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;A concrete example of this is during applyOps failure:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/src/mongo/s/catalog/sharding_catalog_client_impl.cpp#L1325-L1335&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/r3.4.0-rc3/src/mongo/s/catalog/sharding_catalog_client_impl.cpp#L1325-L1335&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The issue here is that when the applyOps fail for example when the current primary steps down, we don&apos;t know how far the write was applied. In the case that the write was replicated, the applyOps will fail when a retry is made because the precondition will fail. However, if we try to inspect the document, we may not see the post-write state because it is not yet in the committed snapshot.&lt;/p&gt;

&lt;p&gt;A fix was attempted before (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20487&quot; title=&quot;w:majority writes that return an error should wait until previous write majority-commits&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20487&quot;&gt;&lt;del&gt;SERVER-20487&lt;/del&gt;&lt;/a&gt;) to always advance the lastVisibleOpTime whenever a write is attempted so the client can use it to do readAfterOpTime. This however will become an issue when the set loses the primary and never advances the opTime (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-24630&quot; title=&quot;Mongos erroneously advances config optime for writes that fail write concern&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-24630&quot;&gt;&lt;del&gt;SERVER-24630&lt;/del&gt;&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;One proposed solution is a hybrid where instead of advancing the global config optime, have the current request either use the last returned visibleOpTime on the readAfterOpTime on the next query or make it wait for replication using getLastError with the returned visibleOpTime.&lt;/p&gt;</description>
                <environment></environment>
        <key id="331342">SERVER-27005</key>
            <summary>Write error revalidate logic needs to wait for lastVisibleOpTime to be committed</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="nathan.myers">Nathan Myers</assignee>
                                    <reporter username="randolph@mongodb.com">Randolph Tan</reporter>
                        <labels>
                    </labels>
                <created>Fri, 11 Nov 2016 19:43:07 +0000</created>
                <updated>Sun, 18 Dec 2016 22:44:36 +0000</updated>
                            <resolved>Sun, 18 Dec 2016 22:44:35 +0000</resolved>
                                    <version>3.2.10</version>
                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="1460677" author="renctan" created="Sun, 18 Dec 2016 22:44:22 +0000"  >&lt;p&gt;I see. Closing this ticket then.&lt;/p&gt;</comment>
                            <comment id="1460573" author="kaloian.manassiev" created="Sun, 18 Dec 2016 06:56:49 +0000"  >&lt;p&gt;&lt;tt&gt;PrepareConfigsFailed&lt;/tt&gt; is an error, which may only be returned from SCCC. Precondition failed returns &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/catalog/apply_ops.cpp#L300&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;BadValue&lt;/a&gt;. Therefore we can never enter the branch, which calls &lt;tt&gt;undoDonateChunk&lt;/tt&gt;.&lt;/p&gt;</comment>
                            <comment id="1460541" author="renctan" created="Sun, 18 Dec 2016 00:10:04 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=kaloian.manassiev&quot; class=&quot;user-hover&quot; rel=&quot;kaloian.manassiev&quot;&gt;kaloian.manassiev&lt;/a&gt; I haven&apos;t tried to repro, but the scenario you explained is different from my previous comment.  I saw a similar situation in a v3.2 build failure for split can also happen for moveChunk. This is the scenario in more detail:&lt;/p&gt;

&lt;p&gt;1. applyOps sent to config.&lt;br/&gt;
2. config steps down.&lt;br/&gt;
3. shard retries applyOps.&lt;br/&gt;
4. previous applyOps happen to get replicated to new primary, so applyOps failed with &quot;preCondition failed&quot;.&lt;br/&gt;
5. The validation logic would try to check the config with readConcern majority, but since it has not yet been committed, it will not see the new chunk. And in this case, it will return with the original preCondition failed error (&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.10/src/mongo/s/catalog/replset/catalog_manager_replica_set.cpp#L953&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ref&lt;/a&gt;) &lt;br/&gt;
6. So it goes to the &quot;if (applyOpsStatus == ErrorCodes::PrepareConfigsFailed)&quot; branch then calls undoDonateChunk and not the else branch that fasserts. (&lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.10/src/mongo/db/s/migration_impl.cpp#L490&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;ref&lt;/a&gt;)&lt;/p&gt;</comment>
                            <comment id="1459969" author="kaloian.manassiev" created="Fri, 16 Dec 2016 16:57:17 +0000"  >&lt;p&gt;I looked at the way migration commit works in 3.2 and I can confirm that this will not cause a problem for it. Specifically, the commit works like this:&lt;/p&gt;

&lt;ol&gt;
	&lt;li&gt;Donor shard &apos;majority&apos; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.11/src/mongo/db/s/migration_impl.cpp#L468&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;writes the applyOps&lt;/a&gt; for the chunk hand-off&lt;/li&gt;
	&lt;li&gt;If there is a network error, refresh the chunks list with &apos;majority&apos; read. There are two possible outcomes here:
	&lt;ol&gt;
		&lt;li&gt;The refresh hits an up-to-date config server node and sees the committed chunk. Because this is a &apos;majority&apos; read, it will not be rolled back, so the migration is successful.&lt;/li&gt;
		&lt;li&gt;The refresh hits a stale config server node and does not see the chunk commit. In this case, we will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.11/src/mongo/db/s/migration_impl.cpp#L499&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;b&gt;crash&lt;/b&gt; the shard&lt;/a&gt;.&lt;/li&gt;
	&lt;/ol&gt;
	&lt;/li&gt;
&lt;/ol&gt;


&lt;p&gt;For the split/merge cases like &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=renctan&quot; class=&quot;user-hover&quot; rel=&quot;renctan&quot;&gt;renctan&lt;/a&gt; mentioned above, this situation is benign because it does not impact routing or filtering and subsequent metadata operations will perform a full refresh with a most up-to-date optime.&lt;/p&gt;

&lt;p&gt;Given that this is not a correctness problem (see above) and that there is only one Evergreen build failure report I am going to close this as Won&apos;t Fix.&lt;/p&gt;</comment>
                            <comment id="1447855" author="renctan" created="Fri, 2 Dec 2016 16:48:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=schwerin&quot; class=&quot;user-hover&quot; rel=&quot;schwerin&quot;&gt;schwerin&lt;/a&gt; After re-examining this again, I believe this issue is benign in the split/merge case, but can cause lost writes for moveChunk. This is because in this scenario applyOps will be applied successfully on the config servers, while the shard will &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.2.10/src/mongo/db/s/migration_impl.cpp#L490&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;undo the increment on the shard version metadata&lt;/a&gt;. This is bad because any mongos who have not yet seen the new config update will continue on sending writes to the shard although it officially doesn&apos;t own it anymore (and the shard thinks it still owns it).&lt;/p&gt;</comment>
                            <comment id="1432280" author="renctan" created="Fri, 11 Nov 2016 20:06:16 +0000"  >&lt;p&gt;Correction: in split and merge, it happens that a write to the config.changelog happens afterwards on success, so the opTime will have advanced and the top level waitForWriteConcern will properly wait for w: majority.&lt;/p&gt;</comment>
                            <comment id="1432263" author="renctan" created="Fri, 11 Nov 2016 19:54:43 +0000"  >&lt;p&gt;Note: v3.4 has a slightly different issue since the applyOps are now initiated from within the config servers and uses local read concern to revalidate. The issue is that since it does not do any write, it may not wait for the { w: majority } to replicate correctly.&lt;/p&gt;</comment>
                            <comment id="1432261" author="kaloian.manassiev" created="Fri, 11 Nov 2016 19:54:16 +0000"  >&lt;p&gt;Alternatively we can always do a majority write after failure in order to get the most up-to-date committed and visible op times. This is what we do during &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_source_manager.cpp#L347&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;migration commit&lt;/a&gt; in 3.4.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                                                <inwardlinks description="is depended on by">
                                                        </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="294594">SERVER-24630</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="230445">SERVER-20487</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 11 Nov 2016 19:54:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 8 weeks, 3 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>randolph@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 8 weeks, 3 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>26.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>kaloian.manassiev@mongodb.com</customfieldvalue>
            <customfieldvalue>nathan.myers</customfieldvalue>
            <customfieldvalue>randolph@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjr1z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsrszr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1316">Sharding 2017-01-02</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsecz3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>