<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:50:56 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-39074] Update outside transaction erasing update from committed prepared transaction</title>
                <link>https://jira.mongodb.org/browse/SERVER-39074</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In the attached concurrency workload, multi-statement transactions simulate transferring money between accounts by subtracting some amount from a document&apos;s &quot;balance&quot; field and adding it to another document&apos;s balance, periodically asserting the sum of all balances never changes (through a collection scan in a transaction with snapshot read concern).&lt;/p&gt;

&lt;p&gt;The workload also concurrently updates other fields in these documents outside of a transaction and when run against a sharded cluster, it fails because the sum of all balances does change, implying one of the updates from a committed transaction was lost. Every thread observes the unexpected sum, so this doesn&apos;t seem to be an issue with reading at the wrong timestamp, and the non-transaction updates don&apos;t modify the &quot;balance&quot; field and use &lt;tt&gt;$set&lt;/tt&gt;, not replacement updates.&lt;/p&gt;

&lt;p&gt;This failure goes away if updates set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; (currently set to true &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a47e42994c06fc500bdcd0bf53d99e199ad5117c/src/mongo/db/read_concern_mongod.cpp#L340&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;), but from talking with the replication team, they don&apos;t believe this should be required for local updates. If that&apos;s true, then it&apos;s possible the non-transaction updates are somehow overwriting the updates of committed prepared transactions.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_62_64_bit_concurrency_sharded_replication_with_balancer_patch_fbc1a0f4c1c34be2e0ba390b750144b4c7919061_5c40ffd7850e61113b894a6d_19_01_17_22_21_36##%257B%2522compare%2522%253A%255B%257B%2522hash%2522%253A%2522fbc1a0f4c1c34be2e0ba390b750144b4c7919061%2522%257D%255D%257D&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Example failure in evergreen&lt;/a&gt;.&lt;/p&gt;</description>
                <environment></environment>
        <key id="674442">SERVER-39074</key>
            <summary>Update outside transaction erasing update from committed prepared transaction</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="louis.williams@mongodb.com">Louis Williams</assignee>
                                    <reporter username="jack.mulrow@mongodb.com">Jack Mulrow</reporter>
                        <labels>
                            <label>prepare_basic</label>
                            <label>txn_storage</label>
                    </labels>
                <created>Thu, 17 Jan 2019 22:07:27 +0000</created>
                <updated>Sun, 29 Oct 2023 22:25:00 +0000</updated>
                            <resolved>Tue, 19 Mar 2019 21:02:05 +0000</resolved>
                                                    <fixVersion>4.1.10</fixVersion>
                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>24</watches>
                                                                                                                <comments>
                            <comment id="2185364" author="xgen-internal-githook" created="Tue, 19 Mar 2019 21:01:21 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;louis.williams@mongodb.com&apos;, &apos;name&apos;: &apos;Louis Williams&apos;, &apos;username&apos;: &apos;louiswilliams&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-39074&quot; title=&quot;Update outside transaction erasing update from committed prepared transaction&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-39074&quot;&gt;&lt;del&gt;SERVER-39074&lt;/del&gt;&lt;/a&gt; All operations enforce prepare conflicts by default&lt;/p&gt;

&lt;p&gt;Prepare conflicts may only be safely ignored when a command can&lt;br/&gt;
guarantee it does not perform writes. Prepare conflicts are ignored when&lt;br/&gt;
the read concern is local, available, or majority and the command is&lt;br/&gt;
aggregate, count, distinct, find, getMore, or group. Aggregate is a&lt;br/&gt;
special case because it may perform writes to an output collection, but&lt;br/&gt;
it enables prepare conflict enforcement before doing so.&lt;/p&gt;

&lt;p&gt;Additionally, connections from a DBDirectClient inherit the&lt;br/&gt;
ignore_prepare state from their parent operation.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/197233a97c2a8859b82ba1ffeac97ba2719f6470&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/197233a97c2a8859b82ba1ffeac97ba2719f6470&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2177643" author="judah.schvimer" created="Mon, 11 Mar 2019 20:22:44 +0000"  >&lt;p&gt;One addendum: We discussed keeping the even stronger invariant that we cannot change &lt;tt&gt;ignorePrepared&lt;/tt&gt; at all, but never setting &lt;tt&gt;ignorePrepared&lt;/tt&gt; in &lt;tt&gt;DBDirectClient&lt;/tt&gt; at all, true or false, and letting it inherit the parent value (or the default if none was set). We likely can skip &lt;tt&gt;waitForReadConcern&lt;/tt&gt; entirely in &lt;tt&gt;DBDirectClient&lt;/tt&gt; but that is out of scope of this work.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; suggested also making &lt;tt&gt;ignorePrepared&lt;/tt&gt; a &lt;tt&gt;boost::optional&lt;/tt&gt; and just using the storage-engine default if none needs to explicitly be specified based on the user provided read concern.&lt;/p&gt;</comment>
                            <comment id="2177582" author="samy.lanka" created="Mon, 11 Mar 2019 19:39:24 +0000"  >&lt;p&gt;Because of concerns surrounding &lt;tt&gt;ignorePrepared&lt;/tt&gt; being set to true by default, we have come up with a new approach. &lt;/p&gt;

&lt;p&gt;We will set the default for &lt;tt&gt;ignorePrepared&lt;/tt&gt; to be false (meaning by default we will enforce prepare conflicts). Specific read commands with readConcern local, majority or available will set &lt;tt&gt;ignorePrepared&lt;/tt&gt; to be true. The commands will be &lt;tt&gt;find&lt;/tt&gt;, &lt;tt&gt;aggregate&lt;/tt&gt;, &lt;tt&gt;count&lt;/tt&gt;, &lt;tt&gt;group&lt;/tt&gt; and &lt;tt&gt;distinct&lt;/tt&gt;, but we will not allow DB direct clients to set &lt;tt&gt;ignorePrepared&lt;/tt&gt; to true. This solution will still require the aggregation work around described above (setting &lt;tt&gt;ignorePrepared&lt;/tt&gt; to true during the read stages and to false during the write stage). We will also keep the invariant that we cannot change &lt;tt&gt;ignorePrepared&lt;/tt&gt; from true to false during an open storage transaction.&lt;/p&gt;</comment>
                            <comment id="2161757" author="samy.lanka" created="Mon, 25 Feb 2019 17:23:25 +0000"  >&lt;blockquote&gt;
&lt;p&gt;What commands besides&#160;&lt;tt&gt;mapReduce&lt;/tt&gt;?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;tt&gt;mergeAuthzCollections&lt;/tt&gt; is the only other command.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Also is WT going to provide an error if we try to ignore a prepare conflict in a transaction that does a write?&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;Yes, I believe that what is described in &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4580&quot; title=&quot;Abort transactions that perform updates with ignore_prepare=true &quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4580&quot;&gt;&lt;del&gt;WT-4580&lt;/del&gt;&lt;/a&gt; should work for this solution.&lt;/p&gt;</comment>
                            <comment id="2161672" author="judah.schvimer" created="Mon, 25 Feb 2019 16:21:50 +0000"  >&lt;p&gt;This sounds reasonable to me.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;There are a couple of commands that still fail the invariant because they use DBDirectClient to do reads and writes on the same storage transaction, so we&apos;ll be explicitly splitting those storage transactions by calling abandonSnapshot on the recovery unit.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;What commands besides &lt;tt&gt;mapReduce&lt;/tt&gt;?&lt;/p&gt;

&lt;p&gt;Also is WT going to provide an error if we try to ignore a prepare conflict in a transaction that does a write?&lt;/p&gt;</comment>
                            <comment id="2161646" author="samy.lanka" created="Mon, 25 Feb 2019 16:06:25 +0000"  >&lt;p&gt;There is actually a problem with the solution proposed above. I ran a patch build after implementing the whitelist for commands that set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; and adding an invariant to make sure that we never try to change &lt;tt&gt;ignore_prepare&lt;/tt&gt; after a storage transaction is started. This invariant was triggered quite often in the patch build due to the command whitelist. This is a problem because changing ignore_prepare after a storage transaction is started won&apos;t actually change the behavior of whether prepare conflicts are ignored or not.&lt;/p&gt;

&lt;p&gt;The new idea for approaching this ticket is to use each command&apos;s &lt;tt&gt;supportsWriteConcern&lt;/tt&gt; function to determine if it performs a write or not, and if it does, set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt;. We will approach aggregate pipelines the same as above by setting &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; during read stages and only setting &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; during the write stage. This will allow the reads to not block on prepare conflict, but make sure that the writes do not overwrite any writes from a prepared transaction. We also want to make sure that index creation will not block on prepared transactions as that would drastically increase the time taken to build an index. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt; suggested that for now we could set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; for &lt;tt&gt;createIndex&lt;/tt&gt; and a change could be made to index builds on a separate ticket to make sure that they won&apos;t block on prepare conflicts.&lt;/p&gt;

&lt;p&gt;Does anyone know if there are any other commands that do writes where setting &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; would cause problems (like unnecessarily blocking reads on prepare conflicts)? I&apos;m not sure if this could happen with &lt;tt&gt;collMod&lt;/tt&gt; or &lt;tt&gt;convertToCapped&lt;/tt&gt;.&lt;/p&gt;

&lt;p&gt;This solution also hits the invariant that we don&apos;t change ignore_prepare after a storage transaction is started, but much less than the other solution. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;&#160;and I decided that we could relax the invariant to only check if we are trying to change ignore_prepare to false after a storage transaction is started, since there isn&apos;t a correctness problem if we don&apos;t ignore prepared transactions. There are a couple of commands that still fail the invariant because they use &lt;tt&gt;DBDirectClient&lt;/tt&gt;&#160;to do reads and writes on the same storage transaction, so we&apos;ll be explicitly splitting those storage transactions by calling &lt;tt&gt;abandonSnapshot&lt;/tt&gt; on the recovery unit.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt; Does this seem reasonable?&lt;/p&gt;</comment>
                            <comment id="2153104" author="schwerin" created="Fri, 15 Feb 2019 13:44:44 +0000"  >&lt;p&gt;What about insert and delete?&lt;/p&gt;

&lt;p&gt;On Thu, Feb 14, 2019, 3:57 PM Samyukta Lanka (Jira) &amp;lt;jira@mongodb.org&amp;gt;&lt;/p&gt;
</comment>
                            <comment id="2153102" author="schwerin" created="Fri, 15 Feb 2019 13:42:03 +0000"  >&lt;p&gt;Insert, too?&lt;/p&gt;</comment>
                            <comment id="2150153" author="samy.lanka" created="Thu, 14 Feb 2019 20:56:56 +0000"  >&lt;p&gt;After talking with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=charlie.swanson&quot; class=&quot;user-hover&quot; rel=&quot;charlie.swanson&quot;&gt;charlie.swanson&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;, we decided on creating a whitelist for write commands that need to set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt;. The whitelisted commands will be &lt;tt&gt;update&lt;/tt&gt;, &lt;tt&gt;find_and_modify&lt;/tt&gt;, &lt;tt&gt;remove&lt;/tt&gt;, &lt;tt&gt;do_txn&lt;/tt&gt; and &lt;tt&gt;apply_ops&lt;/tt&gt;. In aggregate pipelines with $out, we will set &lt;tt&gt;ignore_prepare=false&lt;/tt&gt; only for writes and keep &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; when reading.&lt;/p&gt;

&lt;p&gt;This means that for &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4580&quot; title=&quot;Abort transactions that perform updates with ignore_prepare=true &quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4580&quot;&gt;&lt;del&gt;WT-4580&lt;/del&gt;&lt;/a&gt;, we would need to guarantee that updates are not allowed after skipping prepared updates with &lt;tt&gt;ignore_prepare=true,&lt;/tt&gt;&#160;because we won&apos;t be able to guarantee that no updates are allowed while &lt;tt&gt;ignore_prepare=true&lt;/tt&gt;.&lt;/p&gt;</comment>
                            <comment id="2147981" author="louis.williams" created="Wed, 13 Feb 2019 22:14:24 +0000"  >&lt;p&gt;Discussed with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; offline. He believed that WiredTiger would make the strong guarantee that no updates are allowed while ignore_prepare=true. My conclusion from this discussion, and the ticket I filed for &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4580&quot; title=&quot;Abort transactions that perform updates with ignore_prepare=true &quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4580&quot;&gt;&lt;del&gt;WT-4580&lt;/del&gt;&lt;/a&gt;, was for the weaker guarantee that updates are not allowed after skipping prepared updates with ignore_prepare=true. I think the WiredTiger team can decide which route makes the most sense, and faster performance-wise, because MongoDB will stop performing any writes while ignore_prepare=true.&lt;/p&gt;

&lt;p&gt;With either of these changes, we should still be able to add an invariant in &lt;tt&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/concurrency/write_conflict_exception.h#L93&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;writeConflictRetry&lt;/a&gt;&lt;/tt&gt; that an operation that receives a WCE must not have &lt;tt&gt;RecoveryUnit::getIgnorePrepared() == true&lt;/tt&gt;, a function we will need to add.&lt;/p&gt;</comment>
                            <comment id="2147823" author="judah.schvimer" created="Wed, 13 Feb 2019 20:17:49 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, would returning a WT_ROLLBACK error lead to an infinite write-conflict retry loop, since an update in a prepared transaction will never be able to succeed?&lt;/p&gt;</comment>
                            <comment id="2147674" author="milkie" created="Wed, 13 Feb 2019 18:59:13 +0000"  >&lt;p&gt;Specifically, the ticket says for the update to return WT_ROLLBACK status.  This would be handled the same way we handle WT_ROLLBACK errors today, which is to throw a WriteConflict exception.  It would then be handled by various logic and either retried or returned to the user, depending on the conditions.&lt;/p&gt;</comment>
                            <comment id="2147301" author="judah.schvimer" created="Wed, 13 Feb 2019 15:38:20 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4580&quot; title=&quot;Abort transactions that perform updates with ignore_prepare=true &quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4580&quot;&gt;&lt;del&gt;WT-4580&lt;/del&gt;&lt;/a&gt; calls for the update to fail, not an invariant. How will that manifest to a user if we have a bug that hits that?&lt;/p&gt;</comment>
                            <comment id="2146776" author="louis.williams" created="Wed, 13 Feb 2019 00:43:51 +0000"  >&lt;p&gt;My original suggestion to use an invariant in MongoDB won&apos;t work because it&apos;s legal to call RecoveryUnit::commit() without having performed any writes. I&apos;ve opened &lt;a href=&quot;https://jira.mongodb.org/browse/WT-4580&quot; title=&quot;Abort transactions that perform updates with ignore_prepare=true &quot; class=&quot;issue-link&quot; data-issue-key=&quot;WT-4580&quot;&gt;&lt;del&gt;WT-4580&lt;/del&gt;&lt;/a&gt; for the fix in WiredTiger.&lt;/p&gt;</comment>
                            <comment id="2144915" author="judah.schvimer" created="Mon, 11 Feb 2019 19:29:00 +0000"  >&lt;p&gt;This work should test that all mongodb-transactions set &lt;tt&gt;ignorePrepared = false&lt;/tt&gt; since all mongodb-transactions could do a write. This should already be the behavior, but tests to ensure that behavior is correct and maintained would be good.&lt;/p&gt;</comment>
                            <comment id="2129867" author="alexander.gorrod" created="Mon, 28 Jan 2019 21:18:23 +0000"  >&lt;p&gt;Sounds good - let me know if it makes sense to move the check inside WiredTiger after closer inspection.&lt;/p&gt;</comment>
                            <comment id="2127471" author="judah.schvimer" created="Fri, 25 Jan 2019 16:10:57 +0000"  >&lt;p&gt;Thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt;. If it&apos;s in the mongodb layer, the replication team can do it as part of this ticket with your team&apos;s guidance and review.&lt;/p&gt;</comment>
                            <comment id="2127426" author="louis.williams" created="Fri, 25 Jan 2019 16:05:32 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; Yes, I think it would be easy to, and we should, invariant in the WiredTigerRecoveryUnit that a transaction does not commit with &lt;tt&gt;_ignorePrepared == kIgnore&lt;/tt&gt;. &lt;/p&gt;</comment>
                            <comment id="2127202" author="judah.schvimer" created="Fri, 25 Jan 2019 14:41:34 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.gorrod&quot; class=&quot;user-hover&quot; rel=&quot;alexander.gorrod&quot;&gt;alexander.gorrod&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, that sounds good to me as well. My one concern is that this means if we miss a case then we risk a data corruption bug. Would it be worthwhile/easy enough to add an invariant that we do not do a write in an &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; storage-transaction?&lt;/p&gt;</comment>
                            <comment id="2127190" author="tess.avitabile" created="Fri, 25 Jan 2019 14:34:20 +0000"  >&lt;p&gt;Yes, that sounds good to me.&lt;/p&gt;</comment>
                            <comment id="2126860" author="alexander.gorrod" created="Fri, 25 Jan 2019 04:35:39 +0000"  >&lt;p&gt;I wanted to summarize my reading of the conclusion above: &lt;/p&gt;

&lt;p&gt;MongoDB will stop doing updates from transactions that set &lt;tt&gt;ignore_prepared=true&lt;/tt&gt;. If there is a need to change that in the future - you will let the storage engines team know and we can schedule the work in WiredTiger to cause any transaction with &lt;tt&gt;ignore_prepared=true&lt;/tt&gt; that skips a prepared update to abort as soon as it does an update.&lt;/p&gt;

&lt;p&gt;Does that match your understanding?&lt;/p&gt;</comment>
                            <comment id="2123084" author="judah.schvimer" created="Tue, 22 Jan 2019 18:33:28 +0000"  >&lt;p&gt;If we ran transactions with weaker read concern, writes to prepared documents in the transaction would need to either block or abort the transaction with a WCE. I don&apos;t see how either of those gets more complicated than what we already have. I guess an alternative would be to get the behavior this ticket is describing when using weaker read concerns.&lt;/p&gt;</comment>
                            <comment id="2123062" author="schwerin" created="Tue, 22 Jan 2019 18:21:22 +0000"  >&lt;p&gt;Might  that get complicated is we want to start running MongoDB&lt;br/&gt;
transactions with weaker readconcern?&lt;/p&gt;

&lt;p&gt;On Tue, Jan 22, 2019, 11:01 AM Judah Schvimer (Jira) &amp;lt;jira@mongodb.org&amp;gt;&lt;/p&gt;
</comment>
                            <comment id="2122801" author="judah.schvimer" created="Tue, 22 Jan 2019 16:00:55 +0000"  >&lt;p&gt;&lt;tt&gt;ignore_prepared=false&lt;/tt&gt; is currently only used by linearizable, snapshot, and &lt;tt&gt;afterClusterTime&lt;/tt&gt; reads (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36382&quot; title=&quot;only snapshot, linearizable, and afterClusterTime reads should block on prepare conflicts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36382&quot;&gt;&lt;del&gt;SERVER-36382&lt;/del&gt;&lt;/a&gt;). It was my understanding that any other read concern would be able to (and thus want to for improved performance) see the pre-image of the prepared transaction. local and majority read concern transactions, currently set &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; since that decision happens after we upconvert to snapshot read concern.&lt;/p&gt;

&lt;p&gt;Updates don&apos;t have a read concern, so they default to &lt;tt&gt;ignore_prepared=true&lt;/tt&gt;. I thought they would receive a WCE, but that was a misunderstanding. We could make CRUD ops that do writes that could conflict with prepared transactions (updates, deletes, findAndModify) set &lt;tt&gt;ignore_prepared=false&lt;/tt&gt;. I think that would be sufficient since locking takes care of DDL ops.&lt;/p&gt;</comment>
                            <comment id="2122711" author="tess.avitabile" created="Tue, 22 Jan 2019 15:24:43 +0000"  >&lt;p&gt;I think that either solution &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; described would be correct: (1) transactions that ignored a prepared update must not commit, or (2) transactions with &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; must not perform writes. I don&apos;t think we can assume that mixing local updates with sharded transactions will be rare, so it might be worth considering performance, and I buy &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;&apos;s argument that (2) will be more performant.&lt;/p&gt;</comment>
                            <comment id="2122665" author="schwerin" created="Tue, 22 Jan 2019 15:02:53 +0000"  >&lt;p&gt;I suspect it is a bug in repl to use &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; in updates and deletes. I think it falls out of the behavior of local and majority read concern, which is probably treated the same for reads and writes at present. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; .&lt;/p&gt;</comment>
                            <comment id="2121610" author="michael.cahill" created="Mon, 21 Jan 2019 04:08:18 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, at this point I haven&apos;t tried to lay out a resolution because like you I&apos;m not sure I understand the intent.  We could indeed flag WT transactions so that if they ignore a prepared update, they cannot go on to commit.&lt;/p&gt;

&lt;p&gt;I hope that mixing local updates with sharded transactions isn&apos;t common enough for performance considerations to determine the solution.  Instead, I&apos;m inclined to make it difficult for applications to lose writes when using default transaction and update semantics.&lt;/p&gt;

&lt;p&gt;Let&apos;s wait to hear from the Replication team (and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=schwerin&quot; class=&quot;user-hover&quot; rel=&quot;schwerin&quot;&gt;schwerin&lt;/a&gt;).  If the intent is to have &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; as a common case for updates (as opposed to only enabled for &quot;available&quot; read concern), then I think we need to address this in the storage engine because I don&apos;t see any way for MongoDB to efficiently figure out that these operations have conflicted.&lt;/p&gt;</comment>
                            <comment id="2121607" author="daniel.gottlieb@10gen.com" created="Mon, 21 Jan 2019 03:41:21 +0000"  >&lt;p&gt;I&apos;m not sure what the right compromise is; MongoDBs intentions for using &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; (for things outside of initial sync table scans) is not entirely clear to me.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;[If an &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; transaction can make updates] and it attempts to update a document with a prepared update, is the expectation that it should fail with a&#160;&lt;tt&gt;WT_ROLLBACK&lt;/tt&gt;&#160;at that point?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Can you clarify your statement some? In this example, at the time of the update, the document is no longer prepared. MongoDB making a derivative change relative to the preimage of a prepared update caused the problem. I think the resolution you&apos;re contemplating is:&lt;/p&gt;

&lt;p&gt;If a writing WT transaction ever reads a document where &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; results in a returning a document (as opposed to a prepare conflict), then the WT transaction should return WT_ROLLBACK at its next earliest opportunity. Even if the write occurred before the offending read.&lt;/p&gt;

&lt;p&gt;I&apos;m not sure about the merits of MongoDB relying on such a contract. On the one hand I&apos;ve heard MongoDB&apos;s strategy is to have prepared conflicts block inside storage because storage can be optimized for knowing when the conflict is resolved. However, using &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; and relying on storage to bubble up a WCE just pushes the retrying to the client (where over-waiting introduces latency and under-waiting burns cycles, assuming the client still cares about performing the transaction).&lt;/p&gt;

&lt;p&gt;But perhaps codifying that resolution would be a relatively cheap way to enforce updates don&apos;t get dropped. Thoughts &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;?&lt;/p&gt;</comment>
                            <comment id="2121568" author="michael.cahill" created="Mon, 21 Jan 2019 00:59:57 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, that explanation seems likely.  I didn&apos;t expect &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; transactions to perform writes (because as you explain, they ignore updates by design and therefore can cause lost writes).&lt;/p&gt;

&lt;p&gt;I vaguely remember some conversations about such transactions always being read-only, but clearly that didn&apos;t become part of the interface.&lt;/p&gt;

&lt;p&gt;What is the expected behavior here: should a transaction configured with &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; ever be permitted to perform updates?  If so, and it attempts to update a document &lt;del&gt;with&lt;/del&gt; that already has a prepared update, is the expectation that it should fail with a &lt;tt&gt;WT_ROLLBACK&lt;/tt&gt; at that point?&lt;/p&gt;</comment>
                            <comment id="2120895" author="daniel.gottlieb@10gen.com" created="Fri, 18 Jan 2019 20:48:02 +0000"  >&lt;p&gt;I have a hypothesis of what&apos;s happening. I believe the layer above storage implementing multi-statement transactions depends on the following storage engine behavior/contract:&lt;br/&gt;
When a WT-transaction using &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; returns a pre-image instead of blocking, it must be the case that writing to that document in the same transaction will fail with a WriteConflictException.&lt;/p&gt;

&lt;p&gt;However, I don&apos;t think WiredTiger satisfies that property. I can&apos;t say if this was intentional (i.e: perhaps WT transactions using &lt;tt&gt;ignore_prepare=true&lt;/tt&gt; should not be performing writes) or is a bug. The modification I made to my previous example was to commit the prepared transaction prior to the updater doing a write:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Preparer          | Updater                   | Reader      |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|-------------------+---------------------------+-------------|&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Begin             |                           |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Timestamp :read 5 |                           |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Write A 10        |                           |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Prepare 10        |                           |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Begin :ignorePrepare true |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Timestamp :read 15        |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Read A (WT_NOTFOUND)      |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Commit :commit 12 |                           |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Read A (10)               |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Write A 20                |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Commit :commit 20         |             |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   |                           | Begin       |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   |                           | Read A (20) |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;WT&apos;s rule for returning &lt;tt&gt;WT_ROLLBACK&lt;/tt&gt; (a WriteConflictException) is roughly &quot;If a writer wishes to modify a document, all updates to that document must be visible to the writer for the update to succeed.&quot; In this sequence, by committing the prepared transaction, the updater is able to see the new copy of the update, which implies its write can succeed (and is demonstrated).&lt;/p&gt;

&lt;p&gt;Traditionally, due to the overlap between the Begin/Commit on the &quot;Preparer&quot; and &quot;Updater&quot; sessions, the updater&apos;s write would get a WCE. However, whether intentional or not, the act of a transaction going from a prepared -&amp;gt; committed state changes the evaluation of whether the transactions are considered concurrent.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.gorrod&quot; class=&quot;user-hover&quot; rel=&quot;alexander.gorrod&quot;&gt;alexander.gorrod&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=vamsi.krishna&quot; class=&quot;user-hover&quot; rel=&quot;vamsi.krishna&quot;&gt;vamsi.krishna&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=michael.cahill&quot; class=&quot;user-hover&quot; rel=&quot;michael.cahill&quot;&gt;michael.cahill&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2119974" author="daniel.gottlieb@10gen.com" created="Fri, 18 Jan 2019 04:20:07 +0000"  >&lt;p&gt;Because the bug seems to go away with &lt;tt&gt;ignore_prepare=false&lt;/tt&gt;, I suspect WT/&lt;a href=&quot;https://github.com/mongodb/mongo/blob/bfb0aec447f39633694dcc4418f2a5a5a167937e/src/mongo/db/storage/wiredtiger/wiredtiger_prepare_conflict.h#L65-L74&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Storage Glue&lt;/a&gt; is correctly identifying which documents are in a prepare state. Because the manifestation described above can be explained by Txn #3 seeing the pre-image of a document currently prepared by Txn #2, I&apos;d start looking more into how &lt;tt&gt;writeConflictRetry&lt;/tt&gt; (the exception thrown in response to WT_ROLLBACK, which should* be happening as per the table in my first comment) could be interfering with a multi-statement transaction.&lt;/p&gt;

&lt;p&gt;It might be interesting to further enhance the repro by having the transactions do a read on the documents they are going to update, and putting the contents of that read into a &quot;preimage&quot; field.&lt;/p&gt;</comment>
                            <comment id="2119971" author="daniel.gottlieb@10gen.com" created="Fri, 18 Jan 2019 04:06:18 +0000"  >&lt;p&gt;I looked into this a bit more, reproducing the error. I added logging to the clients to have a second log to reconcile against and determine which document(s) became corrupt. Then I checked the oplog results to ensure there were no updates to a document between prepare/commit (there weren&apos;t) and to determine which transaction(s) went bad. What I found:&lt;/p&gt;

&lt;p&gt;In my repro run, there was $7 more in total than expected. The $7 was entirely attributed to one account (and, from my perspective, one occurrence of the bug). The first three transactions (from the client logs) against the account (all debits/subtractions) were for $153, $7 and $157. The first three versions of the documents were $1847, $1840 and $1690. The relative differences being $153, $7 and $150. To me that means that the third transaction calculated its &lt;tt&gt;&quot;$inc: -157&quot;&lt;/tt&gt; against the $1847 version, instead of the $1840 version.&lt;/p&gt;

&lt;p&gt;As for the additional, irrelevant updates this repro does, there were updates between all of the transactional writes. Furthermore, the first two transactions were prepared while the third one was a single-shard transaction.&lt;/p&gt;</comment>
                            <comment id="2119916" author="daniel.gottlieb@10gen.com" created="Fri, 18 Jan 2019 02:00:53 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams&quot;&gt;louis.williams&lt;/a&gt; pointed me to this and I wanted to rule out that WT had a simple oversight regarding &lt;tt&gt;ignore_prepare&lt;/tt&gt;. Specifically, I was questioning whether using &lt;tt&gt;ignore_prepare&lt;/tt&gt; in a WT transaction writing to some document &lt;tt&gt;D&lt;/tt&gt; would have its write succeed despite &lt;tt&gt;D&lt;/tt&gt; being in a prepared state. I&apos;ve tried a few permutations of interleaving the preparer and updater, but have not been able to produce a wrong result. E.g:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Preparer          | Updater                             | Reader                                  |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|-------------------+-------------------------------------+-----------------------------------------|&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Begin             |                                     |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Timestamp :read 5 |                                     |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Write A 10        |                                     |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Prepare 10        |                                     |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Begin :ignorePrepare true           |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Timestamp :read 15                  |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Write A 20 (-31800 WT_ROLLBACK) [1] |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   | Commit :commit 20                   |                                         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   |                                     | Begin                                   |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                   |                                     | Read A (-31808 WT_PREPARE_CONFLICT) [2] |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Footnotes:&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  [1] conflict between concurrent operations&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;  [2] conflict with a prepared update&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10020">
                    <name>Gantt Dependency</name>
                                            <outwardlinks description="has to be done before">
                                        <issuelink>
            <issuekey id="645266">SERVER-38436</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="688879">WT-4580</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="711094">SERVER-39996</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>33.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 18 Jan 2019 02:00:53 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 47 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1032</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 47 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexander.gorrod@mongodb.com</customfieldvalue>
            <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>michael.cahill@mongodb.com</customfieldvalue>
            <customfieldvalue>samy.lanka@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hujnf3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|huftwf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2783">Repl 2019-02-11</customfieldvalue>
    <customfieldvalue id="2784">Repl 2019-02-25</customfieldvalue>
    <customfieldvalue id="2828">Storage NYC 2019-03-25</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;p&gt;Run this concurrency workload in &lt;tt&gt;concurrency_sharded_replication.yml&lt;/tt&gt; (may need to be repeated):&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;&apos;use strict&apos;;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;load(&apos;jstests/concurrency/fsm_libs/extend_workload.js&apos;);  // for extendWorkload&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;load(&apos;jstests/concurrency/fsm_workloads/multi_statement_transaction_simple.js&apos;);  // for $config&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;var $config = extendWorkload($config, function($config, $super) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    $config.threadCount = 20;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    $config.iterations = 100;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    $config.data.counter = 0;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    /**&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;     * Updates a document that may be written to by the transaction run in the base workload.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;     */&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    $config.states.updateTxnDoc = function updateTxnDoc(db, collName) {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        this.counter += 1;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        // Choose a random document that may be written to by the base workload. The base collection&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        // contains documents with _id ranging from 0 to the number of accounts.&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        const transactionDocId = Random.randInt(this.numAccounts);&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        const threadUniqueField = &apos;thread&apos; + this.tid;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        assertWhenOwnColl.writeOK(db[collName].update({_id: transactionDocId},&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;                                                      {$set: {[threadUniqueField]: this.counter}}));&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    };&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    $config.transitions = {&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        init: {transferMoney: 1},&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        transferMoney: {transferMoney: 0.5, checkMoneyBalance: 0.1, updateTxnDoc: 0.4},&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        checkMoneyBalance: {transferMoney: 0.5, updateTxnDoc: 0.5},&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;        updateTxnDoc: {transferMoney: 0.5, updateTxnDoc: 0.5},&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    };&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&amp;nbsp;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;    return $config;&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;});&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huj9of:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>