<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 05:28:40 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-52667] write_ops_exec::performInserts with retryable writes can lose writes and return an out of order results array</title>
                <link>https://jira.mongodb.org/browse/SERVER-52667</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Found via code inspection in a CR.&lt;/p&gt;

&lt;p&gt;&lt;tt&gt;performInserts&lt;/tt&gt; takes as input a &lt;tt&gt;write_ops::Insert&lt;/tt&gt; (a vector of documents) and returns a result that contains a vector of errors (results*). It&apos;s possible for some earlier writes to contain errors while later ones succeed. However, consider the example where an input batch contains 2 writes: {not yet executed insert, already executed insert}. The output results will be for {already executed insert, not yet executed insert}.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-52667?focusedCommentId=3483436&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-3483436&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;This comment&lt;/a&gt; demonstrates a scenario where I believe the bug in &lt;tt&gt;performInserts&lt;/tt&gt; can manifest as a wrong response to a user.&lt;/p&gt;

&lt;p&gt;&lt;del&gt;My understanding is that today we have no bug. In all cases when the function is called, retryable writes are always at the beginning of the batch (I haven&apos;t verified this). But we are adding new code that uses this method and in ways where the writes may be relatively opaque (sourced by the oplog).&lt;/del&gt;&lt;/p&gt;

&lt;p&gt;&lt;del&gt;I think adding an invariant to formalize the contract or being more careful about batching are both acceptable outcomes.&lt;/del&gt;&lt;/p&gt;

&lt;p&gt;Assigned to sharding for a first look as this seems specific to retryable writes.&lt;/p&gt;</description>
                <environment></environment>
        <key id="1536145">SERVER-52667</key>
            <summary>write_ops_exec::performInserts with retryable writes can lose writes and return an out of order results array</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</assignee>
                                    <reporter username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</reporter>
                        <labels>
                            <label>sharding-wfbf-day</label>
                    </labels>
                <created>Fri, 6 Nov 2020 19:02:56 +0000</created>
                <updated>Sun, 29 Oct 2023 22:00:47 +0000</updated>
                            <resolved>Mon, 16 Nov 2020 19:00:05 +0000</resolved>
                                                    <fixVersion>4.9.0</fixVersion>
                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="3492307" author="xgen-internal-githook" created="Mon, 16 Nov 2020 18:25:06 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Daniel Gottlieb&apos;, &apos;email&apos;: &apos;daniel.gottlieb@mongodb.com&apos;, &apos;username&apos;: &apos;dgottlieb&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-52667&quot; title=&quot;write_ops_exec::performInserts with retryable writes can lose writes and return an out of order results array&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-52667&quot;&gt;&lt;del&gt;SERVER-52667&lt;/del&gt;&lt;/a&gt;: Correct performInserts batching.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/e1e8ea257b835c4d8e130205a2fd8297bcccbd47&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/e1e8ea257b835c4d8e130205a2fd8297bcccbd47&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3483436" author="daniel.gottlieb@10gen.com" created="Tue, 10 Nov 2020 15:28:37 +0000"  >&lt;p&gt;Had a conversation with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt; and there might be a bug today, but connecting the dots is a bit tedious and I&apos;m not familiar enough to know if there&apos;s something the system does to prevent this. First to quickly summarize the contract bug with &lt;tt&gt;performInserts&lt;/tt&gt;. The batching code roughly looks like this:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Results[] performInserts(Document[] inputBatch) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    Result[] results;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    Document[] batchToInsert;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    for (auto doc in inputBatch) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;	if (wasAlreadyExecuted(doc)) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;	    results.append(OK);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;	    continue;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;	}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;	batchToInsert.append(doc);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    insert(batchToInsert, results);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Based on the subset of documents where &lt;tt&gt;wasAlreadyExecuted&lt;/tt&gt; returns true, the indexes referring to a document inside &lt;tt&gt;inputBatch&lt;/tt&gt; may not correspond with the index in the &lt;tt&gt;results&lt;/tt&gt; array. In the simplest case, consider a batch of two inserts. The first needs to be processed, but the second was successfully executed and won&apos;t be retried: &lt;tt&gt;[Op0, Op1]&lt;/tt&gt;. The output results will be in reverse order: &lt;tt&gt;[Result(Op1), Result(Op0)]&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;Additionally, when the last document in the input was an already executed retryable write, the pending batch to be inserted is never actually inserted. The results array will omit contain items for only the inserted data. But the response will seem as if everything was successful.&lt;/p&gt;

&lt;p&gt;I believe a case that exercises this bug in sharding is the following:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Perform a retryable batch insert that goes to two shards, A and B.&lt;/li&gt;
	&lt;li&gt;Shard B succeeds inserting all documents.&lt;/li&gt;
	&lt;li&gt;Shard A gets a retryable error and is only partially successful.&lt;/li&gt;
	&lt;li&gt;Before retrying, a chunk migration moves documents that were in the retryable write from B to A.&lt;/li&gt;
	&lt;li&gt;The retryable write is reissued to A, including the new documents that were originally sent to B.&lt;/li&gt;
	&lt;li&gt;&lt;tt&gt;performInserts&lt;/tt&gt; is called on A in a way that can give incorrect output.
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           | S0                                                               | S1                     |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|-------------------------------------------+------------------------------------------------------------------+------------------------|&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Retryable BatchInsert({_id: 0}, {_id: 1}) |                                                                  |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           | Insert({_id: 0}) -&amp;gt; Network Error, not inserted                  |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           |                                                                  | Insert({_id: 1}) -&amp;gt; OK |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| ChunkMigration {_id: 1} from S1 -&amp;gt; S0     |                                                                  |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           | Insert({_id: 1}) -&amp;gt; OK                                           |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           |                                                                  |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Retry BatchInsert({_id: 0}, {_id: 1})     |                                                                  |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           | performInserts([{_id: 0}, {_id: 1}]) -&amp;gt; {_id: 0} is not inserted |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                           | Response received: [{ok: 1, results: [OK]}]                      |                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;&lt;/li&gt;
&lt;/ul&gt;
</comment>
                            <comment id="3482094" author="daniel.gottlieb@10gen.com" created="Mon, 9 Nov 2020 18:12:53 +0000"  >&lt;p&gt;Saving for posterity, but this comment adds little value. See &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-52667?focusedCommentId=3483436&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-3483436&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;the following comment&lt;/a&gt; for a simplified demonstration of the bug.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The perils of static analysis &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; Without a full grasp of the context, it&apos;s hard to assess risk. As an extreme example, there could be code that changed the value of &lt;tt&gt;opCtx-&amp;gt;inMultiDocumentTransaction&lt;/tt&gt;. But that&apos;s not a viable criticism.&lt;/p&gt;

&lt;p&gt;&amp;gt;What does it mean for a batch to contain both a retryable insert and a non-retryable insert?&lt;/p&gt;

&lt;p&gt;Aside from the fact that the API can&apos;t express this (as retryability is noted on the OpCtx), I&apos;ve now come to the understanding that, certainly, a batch of documents coming from a single insert command can only all be retryable, or none.&lt;/p&gt;

&lt;p&gt;However, &lt;tt&gt;write_ops_exec::performInserts&lt;/tt&gt; isn&apos;t aligned specifically with batches from the perspective of the insert command. One could certainly find two oplog entries that represent inserts, one of which was retryable and the other was not. However I now see that in the case of using &lt;tt&gt;performInserts&lt;/tt&gt; to replay oplog entries, the caller must not be passing in inputs (i.e: setting OpCtx state) that dictate the inserts are to be considered retryable writes.&lt;/p&gt;

&lt;p&gt;It&apos;s becoming a little unclear to me whether &lt;tt&gt;write_ops_exec::performInserts&lt;/tt&gt; should really be used to serve both use-cases (writing a batch from an Insert command as well as doing replication). I believe you (Max) also recently had to avoid &lt;tt&gt;performInserts&lt;/tt&gt; because it implicitly created a collection. Another property beneficial for the insert command that&apos;s not as well suited to replication.&lt;/p&gt;

&lt;p&gt;But before I close this ticket, I&apos;d like to at least correct the terminology (which I perhaps simplified to a fault) and double-check that I know all of the assumptions that are made for the function to be correct (i.e: the output &quot;results&quot; index maps to the input batch insert index).&lt;/p&gt;

&lt;p&gt;Apologies for a terse mathematical definition: Let &lt;a href=&quot;https://github.com/mongodb/mongo/blob/7728778a2138dddb951ab87e95c4337d8e653a86/src/mongo/db/ops/write_ops_exec.cpp#L563-L571&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;(P)redicate&lt;/a&gt; be the set of conditions such that a loop iteration hits &lt;a href=&quot;https://github.com/mongodb/mongo/blob/7728778a2138dddb951ab87e95c4337d8e653a86/src/mongo/db/ops/write_ops_exec.cpp#L575&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this emplace_back&lt;/a&gt;. And (N)on-predicate represents when the condition is not held &lt;a href=&quot;https://github.com/mongodb/mongo/blob/7728778a2138dddb951ab87e95c4337d8e653a86/src/mongo/db/ops/write_ops_exec.cpp#L581&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;and the document is added to the batch&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;The function is correct iff the input batch matches the pattern: &lt;tt&gt;P*N*&lt;/tt&gt;. I think there&apos;s agreement on that?&lt;/p&gt;

&lt;p&gt;Jumping a few levels of indirection, I get to &lt;a href=&quot;https://github.com/mongodb/mongo/blob/7728778a2138dddb951ab87e95c4337d8e653a86/src/mongo/db/transaction_participant.cpp#L2348-L2349&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this clause&lt;/a&gt;. It&apos;s not obvious that calling that function in a sequence (even with only increasing values of stmtId) can only return True*False* (relating to a higher level satisfaction of P*N*). I&apos;d be more convinced if that method were doing a &quot;range comparison&quot;, i.e: &quot;all stmtIds &amp;lt;= 100 are executed and none &amp;gt; 100 are executed&quot;.&lt;/p&gt;&lt;/blockquote&gt;</comment>
                            <comment id="3480779" author="max.hirschhorn@10gen.com" created="Sat, 7 Nov 2020 20:09:16 +0000"  >&lt;blockquote&gt;
&lt;p&gt;However, consider the example where an input batch contains 2 documents: {non-retryable insert, retryable insert}. So long as the non-retryable document &lt;a href=&quot;https://github.com/mongodb/mongo/blob/7728778a2138dddb951ab87e95c4337d8e653a86/src/mongo/db/ops/write_ops_exec.cpp#L583-L584&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;does not hit the batching limit&lt;/a&gt; (has its write deferred), the output results will be for {retryable insert, non-retryable insert}.&lt;/p&gt;

&lt;p&gt;My understanding is that today we have no bug. In all cases when the function is called, retryable writes are always at the beginning of the batch (I haven&apos;t verified this). But we are adding new code that uses this method and in ways where the writes may be relatively opaque (sourced by the oplog).&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;What does it mean for a batch to contain both a retryable insert and a non-retryable insert? An insert command contains either non-retryable inserts or retryable inserts (or inserts in a multi-document transaction) depending on the &quot;txnNumber&quot; field being presented in the command arguments. The txnNumber becomes a property of the OperationContext and so a batch of inserts can be executed either only as retryable or only as non-retryable.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10520">
                    <name>Problem/Incident</name>
                                            <outwardlinks description="causes">
                                        <issuelink>
            <issuekey id="1548543">SERVER-52978</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>4.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 7 Nov 2020 20:09:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        3 years, 12 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17050" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Downstream Team Attention</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="16941"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            3 years, 12 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>max.hirschhorn@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hygdnr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy2giv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hyfzx3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>