<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:49:07 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38499] Preparing transaction fails and triggers invariant if chosen timestamp is not greater than WiredTiger&apos;s latest active read timestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-38499</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;To prepare a transaction, a slot is &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a1442e88e7/src/mongo/db/transaction_participant.cpp#L842&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;reserved in the oplog&lt;/a&gt;, and then the slot&apos;s opTime timestamp &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a1442e88e7/src/mongo/db/transaction_participant.cpp#L857-L858&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;is used to prepare the transaction in WiredTiger&lt;/a&gt;. If a new WiredTiger transaction begins during this window (i.e. after reserving a time, before it is given to WT) and starts reading at a timestamp &amp;gt;= the reserved prepare timestamp, WiredTiger will fail to prepare the transaction because &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a1442e88e7/src/third_party/wiredtiger/src/txn/txn_timestamp.c#L698-L701&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&quot;the prepare timestamp must be later/greater than the latest active read timestamp&quot;&lt;/a&gt;, failing &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a1442e88e7/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L185&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;this invariant&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Example crash (from &lt;tt&gt;indexed_insert_large_noindex.js&lt;/tt&gt; failure in this &lt;a href=&quot;https://evergreen.mongodb.com/task/mongodb_mongo_master_linux_64_debug_concurrency_sharded_replication_multi_stmt_txn_with_balancer_patch_b18a6b96d0c28bc98a1e16b7de0d5f104fd3c937_5c096a752a60ed2d89759d97_18_12_06_18_29_29##%257B%2522compare%2522%253A%255B%257B%2522hash%2522%253A%2522b18a6b96d0c28bc98a1e16b7de0d5f104fd3c937%2522%257D%252C%257B%2522hash%2522%253A%2522b18a6b96d0c28bc98a1e16b7de0d5f104fd3c937%2522%257D%255D%257D&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;evergreen patch&lt;/a&gt;):&#160;&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] 2018-12-06T19:01:36.043+0000 E STORAGE  [conn13085] WiredTiger error (22) [1544122896:43869][47134:0x7f8d88774700], WT_SESSION.prepare_transaction: __wt_txn_parse_prepare_timestamp, 722: prepare timestamp 5C09721000000006 not later than an active read timestamp 5c09721000000007 : Invalid argument Raw: [1544122896:43869][47134:0x7f8d88774700], WT_SESSION.prepare_transaction: __wt_txn_parse_prepare_timestamp, 722: prepare timestamp 5C09721000000006 not later than an active read timestamp 5c09721000000007 : Invalid argument&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] 2018-12-06T19:01:36.043+0000 F -        [conn13085] Invariant failure: s-&amp;gt;prepare_transaction(s, conf.c_str()) resulted in status BadValue: 22: Invalid argument at src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp 185&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] 2018-12-06T19:01:36.044+0000 F -        [conn13085]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] ***aborting after invariant() failure&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] &lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] 2018-12-06T19:01:36.075+0000 F -        [conn13085] Got signal: 6 (Aborted).&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  0x7f8dbfb638e1 0x7f8dbfb62af9 0x7f8dbfb62fdd 0x7f8dbc1307e0 0x7f8dbbdbf495 0x7f8dbbdc0c75 0x7f8dbe0d8f79 0x7f8dbe1a31ee 0x7f8dbfa9f6c1 0x7f8dbf07e573 0x7f8dbe394c45 0x7f8dbe395595 0x7f8dbe56e834 0x7f8dbe570ca0 0x7f8dbe5728be 0x7f8dbe57369f 0x7f8dbe56003a 0x7f8dbe56c6be 0x7f8dbe56822f 0x7f8dbe56b63d 0x7f8dbf2dbb42 0x7f8dbe565da8 0x7f8dbe569104 0x7f8dbe5673cc 0x7f8dbe5682c1 0x7f8dbe56b63d 0x7f8dbf2dc0b5 0x7f8dbfa9e9c4 0x7f8dbc128aa1 0x7f8dbbe75bdd&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] ----- BEGIN BACKTRACE -----&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary] {&quot;backtrace&quot;:[{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;24F18E1&quot;,&quot;s&quot;:&quot;_ZN5mongo15printStackTraceERSo&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;24F0AF9&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;24F0FDD&quot;},{&quot;b&quot;:&quot;7F8DBC121000&quot;,&quot;o&quot;:&quot;F7E0&quot;},{&quot;b&quot;:&quot;7F8DBBD8D000&quot;,&quot;o&quot;:&quot;32495&quot;,&quot;s&quot;:&quot;gsignal&quot;},{&quot;b&quot;:&quot;7F8DBBD8D000&quot;,&quot;o&quot;:&quot;33C75&quot;,&quot;s&quot;:&quot;abort&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;A66F79&quot;,&quot;s&quot;:&quot;_ZN5mongo24invariantOKFailedWithMsgEPKcRKNS_6StatusERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;B311EE&quot;,&quot;s&quot;:&quot;_ZN5mongo22WiredTigerRecoveryUnit17prepareUnitOfWorkEv&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;242D6C1&quot;,&quot;s&quot;:&quot;_ZN5mongo15WriteUnitOfWork7prepareEv&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;1A0C573&quot;,&quot;s&quot;:&quot;_ZN5mongo22TransactionParticipant18prepareTransactionEPNS_16OperationContextEN5boost8optionalINS_4repl6OpTimeEEE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;D22C45&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;D23595&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EFC834&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EFECA0&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;F008BE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;F0169F&quot;,&quot;s&quot;:&quot;_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EEE03A&quot;,&quot;s&quot;:&quot;_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EFA6BE&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF622F&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF963D&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;1C69B42&quot;,&quot;s&quot;:&quot;_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF3DA8&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF7104&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF53CC&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF62C1&quot;,&quot;s&quot;:&quot;_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;EF963D&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;1C6A0B5&quot;},{&quot;b&quot;:&quot;7F8DBD672000&quot;,&quot;o&quot;:&quot;242C9C4&quot;},{&quot;b&quot;:&quot;7F8DBC121000&quot;,&quot;o&quot;:&quot;7AA1&quot;},{&quot;b&quot;:&quot;7F8DBBD8D000&quot;,&quot;o&quot;:&quot;E8BDD&quot;,&quot;s&quot;:&quot;clone&quot;}],&quot;processInfo&quot;:{ &quot;mongodbVersion&quot; : &quot;4.1.6-32-gb18a6b96d0-patch-5c096a752a60ed2d89759d97&quot;, &quot;gitVersion&quot; : &quot;b18a6b96d0c28bc98a1e16b7de0d5f104fd3c937&quot;, &quot;compiledModules&quot; : [], &quot;uname&quot; : { &quot;sysname&quot; : &quot;Linux&quot;, &quot;release&quot; : &quot;2.6.32-220.el6.x86_64&quot;, &quot;version&quot; : &quot;#1 SMP Wed Nov 9 08:03:13 EST 2011&quot;, &quot;machine&quot; : &quot;x86_64&quot; }, &quot;somap&quot; : [ { &quot;b&quot; : &quot;7F8DBD672000&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;57732CF5367F00101F6255B61CA5CD0571C83E54&quot; }, { &quot;b&quot; : &quot;7FFF242FF000&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;08F634A1D22DEFF00461D50A7699DACDC97657BF&quot; }, { &quot;b&quot; : &quot;7F8DBD235000&quot;, &quot;path&quot; : &quot;/lib64/libresolv.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;F0BE1166EDCFFB2422B940D601A1BBD89352D80F&quot; }, { &quot;b&quot; : &quot;7F8DBCE50000&quot;, &quot;path&quot; : &quot;/usr/lib64/libcrypto.so.10&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;1EDB45C205A844A75EBBB4F0075E705803FFB85B&quot; }, { &quot;b&quot; : &quot;7F8DBCBE4000&quot;, &quot;path&quot; : &quot;/usr/lib64/libssl.so.10&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;D256E285C5E11D9A99EB04CA7651003A8F67B64E&quot; }, { &quot;b&quot; : &quot;7F8DBC9E0000&quot;, &quot;path&quot; : &quot;/lib64/libdl.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;1F7E85410384392BC51FA7324961719A10125F31&quot; }, { &quot;b&quot; : &quot;7F8DBC7D8000&quot;, &quot;path&quot; : &quot;/lib64/librt.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;FDF3A36FFFE08375456D59DA959EAB2FC30B6186&quot; }, { &quot;b&quot; : &quot;7F8DBC554000&quot;, &quot;path&quot; : &quot;/lib64/libm.so.6&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;8A852AC42F0B64F0F30C760EBBCFA3FE4A228F12&quot; }, { &quot;b&quot; : &quot;7F8DBC33E000&quot;, &quot;path&quot; : &quot;/lib64/libgcc_s.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;EDC925E58FE28DCA536993EB13179C739F1E6566&quot; }, { &quot;b&quot; : &quot;7F8DBC121000&quot;, &quot;path&quot; : &quot;/lib64/libpthread.so.0&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;85104ECFE42C606B31C2D0D0D2E5DACD3286A341&quot; }, { &quot;b&quot; : &quot;7F8DBBD8D000&quot;, &quot;path&quot; : &quot;/lib64/libc.so.6&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;8E3AACE76351B6A83390CA065E904EB82FBD1EC7&quot; }, { &quot;b&quot; : &quot;7F8DBD44F000&quot;, &quot;path&quot; : &quot;/lib64/ld-linux-x86-64.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;1CC2165E019D43F71FDE0A47AF9F4C8EB5E51963&quot; }, { &quot;b&quot; : &quot;7F8DBBB77000&quot;, &quot;path&quot; : &quot;/lib64/libz.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;D053BB4FF0C2FC983842F81598813B9B931AD0D1&quot; }, { &quot;b&quot; : &quot;7F8DBB933000&quot;, &quot;path&quot; : &quot;/lib64/libgssapi_krb5.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;0C249DF4D77989253CCD859956BF50749308A16A&quot; }, { &quot;b&quot; : &quot;7F8DBB64C000&quot;, &quot;path&quot; : &quot;/lib64/libkrb5.so.3&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;624C7056B8BBE6BA758DEF557F516FBDBD01E1FD&quot; }, { &quot;b&quot; : &quot;7F8DBB448000&quot;, &quot;path&quot; : &quot;/lib64/libcom_err.so.2&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;57F77704A7F1F4E3689D028D3F9ADD4E77486EC9&quot; }, { &quot;b&quot; : &quot;7F8DBB21C000&quot;, &quot;path&quot; : &quot;/lib64/libk5crypto.so.3&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;C81673692EEF670BC951EE726490F5D1CAB822F4&quot; }, { &quot;b&quot; : &quot;7F8DBB011000&quot;, &quot;path&quot; : &quot;/lib64/libkrb5support.so.0&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;03B69EEB8998AC9CA7519A27571BAD976BA4C56D&quot; }, { &quot;b&quot; : &quot;7F8DBAE0E000&quot;, &quot;path&quot; : &quot;/lib64/libkeyutils.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;3BCCABE75DC61BBA81AAE45D164E26EF4F9F55DB&quot; }, { &quot;b&quot; : &quot;7F8DBABEF000&quot;, &quot;path&quot; : &quot;/lib64/libselinux.so.1&quot;, &quot;elfType&quot; : 3, &quot;buildId&quot; : &quot;B4576BE308DDCF7BC31F7304E4734C3D846D0236&quot; } ] }}&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x7f8dbfb638e1]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(+0x24F0AF9) [0x7f8dbfb62af9]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(+0x24F0FDD) [0x7f8dbfb62fdd]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  libpthread.so.0(+0xF7E0) [0x7f8dbc1307e0]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  libc.so.6(gsignal+0x35) [0x7f8dbbdbf495]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  libc.so.6(abort+0x175) [0x7f8dbbdc0c75]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(_ZN5mongo24invariantOKFailedWithMsgEPKcRKNS_6StatusERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES1_j+0x0) [0x7f8dbe0d8f79]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(_ZN5mongo22WiredTigerRecoveryUnit17prepareUnitOfWorkEv+0x2DE) [0x7f8dbe1a31ee]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(_ZN5mongo15WriteUnitOfWork7prepareEv+0x31) [0x7f8dbfa9f6c1]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(_ZN5mongo22TransactionParticipant18prepareTransactionEPNS_16OperationContextEN5boost8optionalINS_4repl6OpTimeEEE+0x143) [0x7f8dbf07e573]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(+0xD22C45) [0x7f8dbe394c45]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;[ShardedClusterFixture:job0:shard1:primary]  mongod(+0xD23595) [0x7f8dbe395595]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;</description>
                <environment></environment>
        <key id="648848">SERVER-38499</key>
            <summary>Preparing transaction fails and triggers invariant if chosen timestamp is not greater than WiredTiger&apos;s latest active read timestamp</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="daniel.gottlieb@mongodb.com">Daniel Gottlieb</assignee>
                                    <reporter username="jack.mulrow@mongodb.com">Jack Mulrow</reporter>
                        <labels>
                            <label>prepare_basic</label>
                    </labels>
                <created>Mon, 10 Dec 2018 15:38:41 +0000</created>
                <updated>Sun, 29 Oct 2023 22:25:57 +0000</updated>
                            <resolved>Fri, 25 Jan 2019 01:40:17 +0000</resolved>
                                                    <fixVersion>4.1.8</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>10</watches>
                                                                                                                <comments>
                            <comment id="2126781" author="xgen-internal-githook" created="Fri, 25 Jan 2019 01:39:52 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;username&apos;: &apos;dgottlieb&apos;, &apos;email&apos;: &apos;daniel.gottlieb@mongodb.com&apos;, &apos;name&apos;: &apos;Daniel Gottlieb&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38499&quot; title=&quot;Preparing transaction fails and triggers invariant if chosen timestamp is not greater than WiredTiger&amp;#39;s latest active read timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38499&quot;&gt;&lt;del&gt;SERVER-38499&lt;/del&gt;&lt;/a&gt;: Enforce oplog visibility at the MongoDB layer.&lt;/p&gt;

&lt;p&gt;WiredTiger guards against transactions preparing with a timestamp&lt;br/&gt;
earlier than the most recent reader. This guarantees no reader may&lt;br/&gt;
have seen the wrong version of a document.&lt;/p&gt;

&lt;p&gt;The oplog is a special case. Because the oplog does not contain&lt;br/&gt;
prepared updates, and oplog readers cannot read from other&lt;br/&gt;
collections, it&apos;s valid to prepare behind an oplog readers&lt;br/&gt;
timestamp.&lt;/p&gt;

&lt;p&gt;However, WiredTiger is not aware the oplog is special. When MongoDB&lt;br/&gt;
uses WiredTiger `read_timestamp`s to enforce oplog visibility, there&lt;br/&gt;
are cases (specifically, secondary oplog application) where an oplog&lt;br/&gt;
reader can be in front of an impending prepare.&lt;/p&gt;

&lt;p&gt;There were two strategies available for resolving this. The first is&lt;br/&gt;
to artificially hold back what oplog is available to read at until&lt;br/&gt;
nothing can be prepared behind an oplog reader. The second strategy,&lt;br/&gt;
which is what this patch does, is to have the MongoDB layer hide&lt;br/&gt;
documents that are newer than the visibility point. The mechanism for&lt;br/&gt;
calculating and discovering the visibility point is unchanged.&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/5f213f2d419d9549559281fef7d3704ad7614d12&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/5f213f2d419d9549559281fef7d3704ad7614d12&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="2117648" author="judah.schvimer" created="Wed, 16 Jan 2019 21:49:29 +0000"  >&lt;p&gt;After discussion, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; will make sure that oplog readers don&#8217;t do timestamped reads, so that they don&#8217;t contribute to the latest active read timestamp. This won&#8217;t have any effect on majority commit latency. &lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38906&quot; title=&quot;Multi-document transactions should not perform timestamped read ahead of all-committed&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38906&quot;&gt;&lt;del&gt;SERVER-38906&lt;/del&gt;&lt;/a&gt; will ensure that all reads on a primary that have a timestamp occur behind the &lt;tt&gt;all_committed&lt;/tt&gt;. &lt;/p&gt;

&lt;p&gt;This should enable us to keep the invariant exactly as it exists today. &lt;/p&gt;</comment>
                            <comment id="2113497" author="judah.schvimer" created="Sun, 13 Jan 2019 20:38:16 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=agorrod&quot; class=&quot;user-hover&quot; rel=&quot;agorrod&quot;&gt;agorrod&lt;/a&gt;, do you have any thoughts on the above options, especially regarding relaxing the invariants in WT?&lt;/p&gt;</comment>
                            <comment id="2107761" author="tess.avitabile" created="Mon, 7 Jan 2019 20:29:54 +0000"  >&lt;p&gt;I hadn&apos;t thought about the fact that for Option 7, the invariant would have to differ on primaries and secondaries, so it couldn&apos;t be done in the storage layer without additional inputs. Thank you for thinking it through. Now I don&apos;t like that choice as much. Now I prefer Option 2 again. Or Option 5, but it seems like a lot of work.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; also pointed out that Option 6 would slow majority write acknowledgment even when chaining is not used, since majority acknowledgment does not occur until the write is behind the &lt;tt&gt;all_committed&lt;/tt&gt; on a majority of nodes.&lt;/p&gt;</comment>
                            <comment id="2107654" author="judah.schvimer" created="Mon, 7 Jan 2019 19:51:05 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt;, &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, and I discussed this and here are our thoughts and conclusions. &lt;/p&gt;

&lt;p&gt;There are two problems.&lt;br/&gt;
1) Local and Majority Multi-document Transactions read at &lt;tt&gt;lastApplied&lt;/tt&gt; rather than &lt;tt&gt;all_committed&lt;/tt&gt;, and transactions can be prepared at a timestamp less than the &lt;tt&gt;lastApplied&lt;/tt&gt;, but must be greater than the &lt;tt&gt;all_committed&lt;/tt&gt; since they create an oplog hole themselves.&lt;br/&gt;
2) Oplog readers on secondaries read at &lt;tt&gt;all_committed&lt;/tt&gt;. After writing the oplog entries in a batch, &lt;tt&gt;all_committed&lt;/tt&gt; can advance to the end of the batch. Thus an oplog reader can start reading at a timestamp greater than that of oplog entries that have been written but not applied. One of these oplog entries might try to prepare a transaction at a timestamp in the middle of the batch, even though there is an oplog reader open at the timestamp of the end of the batch.&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; points out that there is no fundamental reason why local and majority multi-document transactions are given a read timestamp. They already do not read at a snapshot (since they do not read at &lt;tt&gt;all_committed&lt;/tt&gt; ) so they may as well read the most recent data like a normal local read. Read-only transactions would just have to consult the system last optime to see what timestamp to wait for write concern on, rather than looking at the read timestamp. This means that on primaries all reads at a timestamp would specify &lt;tt&gt;ignore_prepared=false&lt;/tt&gt;. &lt;/p&gt;

&lt;p&gt;This doesn&apos;t work on secondaries though because oplog readers must read at a timestamp. To solve the secondary case there are a few options.&lt;br/&gt;
1) Relax the invariant completely: Allow a transaction to be prepared behind the read timestamp of any storage transactions. This one is easy but removes a valuable invariant.&lt;br/&gt;
2) Relax the invariant partially (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38832&quot; title=&quot;Allow a transaction to be prepared behind the read timestamp of storage transactions with ignore_prepared=true&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38832&quot;&gt;&lt;del&gt;SERVER-38832&lt;/del&gt;&lt;/a&gt;): Allow a transaction to be prepared behind the read timestamp of storage transactions with &lt;tt&gt;ignore_prepared=true&lt;/tt&gt; (oplog readers set &lt;tt&gt;ignore_prepared=true&lt;/tt&gt;. This is likely a hard invariant to construct requiring tracking extra data.&lt;br/&gt;
3) Relax the invariant on secondaries: Pass a parameter to &lt;tt&gt;prepare_transaction&lt;/tt&gt; that says whether or not we want to check the invariant. The invariant is still something that we want to maintain, but this would be a middle ground between the above 2 options in terms of safety vs. amount of work.&lt;br/&gt;
4) Relax the invariant for oplog readers: This would require oplog readers to specify that they should not be included in the &quot;read timestamp list&quot;. This may have unforeseen side effects and would make the oplog more special for better or for worse.&lt;br/&gt;
5) Maintain two different ideas of &lt;tt&gt;all_committed&lt;/tt&gt;: one for data holes and one for oplog holes. This is a major change, backtracking a bit towards where we were in 3.4.&lt;br/&gt;
6) Delay moving forward &lt;tt&gt;all_committed&lt;/tt&gt; to the end of a batch until the batch has been applied. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; pointed out that this will slow down replication when chaining is in use and is an undesirable solution.&lt;br/&gt;
7) Replace the invariant with &quot;Readers with &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; set must be reading behind the &lt;tt&gt;all_committed&lt;/tt&gt; timestamp&quot;. This somewhat flips the invariant around, putting the onus on the readers rather than on the preparer. On primaries we could add the invariant that &quot;Transactions must be prepared at a timestamp greater than or equal to the &lt;tt&gt;all_committed&lt;/tt&gt; timestamp&quot;. Together these mean that &quot;Prepared transactions must be prepared at a timestamp greater than &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; readers&quot;. On secondaries we could invariant that &quot;Transaction must be prepared at a timestamp greater than or equal to &lt;tt&gt;lastApplied&lt;/tt&gt;&quot; and we could also invariant that on secondaries &quot;Readers with &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; set must be reading behind the &lt;tt&gt;lastApplied&lt;/tt&gt; timestamp&quot;. These two together would mean that Prepared transactions must be prepared at a timestamp greater than &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; readers&quot;.&lt;/p&gt;

&lt;p&gt;Option 7 seems like the best way forward. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, please provide any thoughts.&lt;br/&gt;
CCing &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=agorrod&quot; class=&quot;user-hover&quot; rel=&quot;agorrod&quot;&gt;agorrod&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt; as well.&lt;/p&gt;</comment>
                            <comment id="2105033" author="daniel.gottlieb@10gen.com" created="Fri, 4 Jan 2019 00:00:53 +0000"  >&lt;blockquote&gt;
&lt;p&gt;Daniel Gottlieb, why would oplog readers use snapshot isolation with a timestamp in the middle of the current batch? I would expect them to either read with local read concern or with snapshot isolation at &quot;lastApplied&quot; which would be at the end of the previous batch.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/71f49041edda00fb1ebf224d53a8bf61726f5ac4/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp#L1947&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Oplog readers&lt;/a&gt; are &lt;a href=&quot;https://github.com/mongodb/mongo/blob/71f49041edda00fb1ebf224d53a8bf61726f5ac4/src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp#L482-L487&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;assigned a read timestamp&lt;/a&gt; at the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/71f49041edda00fb1ebf224d53a8bf61726f5ac4/src/mongo/db/storage/wiredtiger/wiredtiger_oplog_manager.cpp#L214-L234&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;no holes point, as tracked by storage&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Secondary oplog application can mess with storage&apos;s notion of the no holes point because transactions don&apos;t begin in timestamp order (compared to primary&apos;s where transactions may not commit in order, a simpler problem).&lt;/p&gt;

&lt;p&gt;I&apos;m still curious if it&apos;s actually necessary to assign a read timestamp to (recovery unit) transactions if those timestamps can be ahead of the no-holes point.&lt;/p&gt;</comment>
                            <comment id="2105004" author="judah.schvimer" created="Thu, 3 Jan 2019 23:22:39 +0000"  >&lt;p&gt;I filed &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38832&quot; title=&quot;Allow a transaction to be prepared behind the read timestamp of storage transactions with ignore_prepared=true&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38832&quot;&gt;&lt;del&gt;SERVER-38832&lt;/del&gt;&lt;/a&gt; to relax the invariant. This ticket will be for the replication work to correctly set &quot;ignore_prepared&quot;.&lt;/p&gt;</comment>
                            <comment id="2104963" author="tess.avitabile" created="Thu, 3 Jan 2019 22:42:00 +0000"  >&lt;p&gt;I believe we also still need to relax the invariant to say that a transaction must not be prepared behind the read timestamp of any storage transaction &lt;em&gt;with ignore_prepared=false&lt;/em&gt;.&lt;/p&gt;</comment>
                            <comment id="2104909" author="judah.schvimer" created="Thu, 3 Jan 2019 22:07:17 +0000"  >&lt;p&gt;Ok, thanks for the correction. So it seems that the only work item here is fixing ignore_prepared for multi-document transactions with local and majority read concerns. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt;, please correct me if I&apos;m missing anything.&lt;/p&gt;</comment>
                            <comment id="2104254" author="tess.avitabile" created="Thu, 3 Jan 2019 15:26:29 +0000"  >&lt;p&gt;For transactions with local and majority read concern, we read from the lastApplied, rather than the all-committed. We set the timestamp read source &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3a90146b8e7a589e902a6cd0bd3520e971e722ce/src/mongo/db/transaction_participant.cpp#L466-L469&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; based on the original read concern, which is obtained &lt;a href=&quot;https://github.com/mongodb/mongo/blob/3a90146b8e7a589e902a6cd0bd3520e971e722ce/src/mongo/db/transaction_participant.cpp#L778-L781&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. It is important that transactions with local and majority read concern read from the lastApplied so that back-to-back transactions can read their own writes, as described in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38204&quot; title=&quot;Remove difference between readConcern:&amp;quot;majority&amp;quot; and readConcern:&amp;quot;snapshot&amp;quot;&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38204&quot;&gt;&lt;del&gt;SERVER-38204&lt;/del&gt;&lt;/a&gt;. I agree that they are not providing snapshot isolation--converting the read concern to snapshot internally is simply convenient for the implementation. Since these transactions can read at a time later than the all-committed, they must have ignore_prepared=true or they will still trigger the invariant after the proposed change.&lt;/p&gt;</comment>
                            <comment id="2103889" author="judah.schvimer" created="Wed, 2 Jan 2019 23:34:24 +0000"  >&lt;p&gt;Ah, you&apos;re referring to multi-document transactions with local read concern, not storage-transactions with local read concern. We would only need to fix that bug when we actually support local and majority read concern for transactions. Right now since everything is upconverted to snapshot, they should be reading at a time earlier than the all-committed. If not, then we&apos;re not actually upconverting the read concern correctly and we&apos;re not actually providing snapshot isolation.&lt;/p&gt;</comment>
                            <comment id="2103860" author="tess.avitabile" created="Wed, 2 Jan 2019 23:09:14 +0000"  >&lt;p&gt;We set ignore_prepared=false &lt;a href=&quot;https://github.com/mongodb/mongo/blob/0f05363c0431e4b5e6e8b4dcc14ad124de387413/src/mongo/db/read_concern_mongod.cpp#L209&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; for snapshot reads. However, this is based on the upconverted read concern, rather than the original readConcern. IIUC, this means we do not ignore prepare conflicts for transactions with readConcern local or majority.&lt;/p&gt;</comment>
                            <comment id="2103854" author="judah.schvimer" created="Wed, 2 Jan 2019 23:05:07 +0000"  >&lt;blockquote&gt;
&lt;p&gt;we will need to fix the bug where transactions reading with local or majority readConcern set ignore_prepared=false&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt;, what bug are you referring to? This sounds like &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36382&quot; title=&quot;only snapshot, linearizable, and afterClusterTime reads should block on prepare conflicts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36382&quot;&gt;&lt;del&gt;SERVER-36382&lt;/del&gt;&lt;/a&gt; which was already fixed.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;oplog readers can slice between the two operations with a read timestamp that&apos;s ahead of the imminent prepare time&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, why would oplog readers use snapshot isolation with a timestamp in the middle of the current batch? I would expect them to either read with local read concern or with snapshot isolation at &quot;lastApplied&quot; which would be at the end of the previous batch.&lt;/p&gt;</comment>
                            <comment id="2103512" author="daniel.gottlieb@10gen.com" created="Wed, 2 Jan 2019 19:27:42 +0000"  >&lt;p&gt;A clarification. After talking with &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt;, we realized the hypothesis for where we had &quot;violating readers&quot; in the system was incomplete. There were test failures on secondaries which I believe have the following sequence:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Oplog Applier        | Oplog Manager                 | Oplog Reader              |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|----------------------+-------------------------------+---------------------------|&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Begin                |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Write Oplog 10       |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Timestamp :commit 10 |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Commit               |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                      | Update oplog visibility to 10 |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                      |                               | Begin :isolation snapshot |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                      |                               | Timestamp :readAt 10      |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Begin                |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Write A 1            |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Prepare 10           |                               |                           |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Due to oplog application first writing out to the oplog followed by applying the entry in separate transactions, oplog readers can slice between the two operations with a read timestamp that&apos;s ahead of the imminent prepare time.&lt;/p&gt;</comment>
                            <comment id="2103213" author="tess.avitabile" created="Wed, 2 Jan 2019 16:17:56 +0000"  >&lt;p&gt;If we relax this invariant, we will need to fix the bug where transactions reading with local or majority readConcern set ignore_prepared=false. Transactions with local or majority readConcern read at the lastApplied, which may be ahead of the all-committed, so a transaction could prepare behind their read timestamp.&lt;/p&gt;</comment>
                            <comment id="2103061" author="daniel.gottlieb@10gen.com" created="Wed, 2 Jan 2019 14:21:46 +0000"  >&lt;p&gt;I don&apos;t think I have full perspective on the problem at this time to recommend that removing/adding precision to the WT error is the right course of action. Creating a precise error check, I believe, adds more computation to a hot critical section. Removing the error check results in difficult bugs to make manifest and diagnose. A third option would be to have the precise error check in WT &quot;diagnostic&quot; builds and no error check otherwise. Though, I&apos;m not sure if that&apos;s what our build configuration does on debug builds today and changing things might discover some complications.&lt;/p&gt;

&lt;p&gt;I accept your statement about why preparing behind some readers that set a read timestamps (with ignore_prepare) is fine, but I haven&apos;t been convinced of the following:&lt;/p&gt;
&lt;ol&gt;
	&lt;li&gt;That the &quot;violating&quot; readers in the test failures were in fact configured with &lt;tt&gt;ignore_prepare=true&lt;/tt&gt;.&lt;/li&gt;
	&lt;li&gt;Why readers using &lt;tt&gt;ignore_prepare=true&lt;/tt&gt;, that want to read in front of a no-holes point, require a read timestamp at all. If repl has already done the appropriate waiting for any causal relationships, does assigning a read timestamp, possibly in front of the no-holes point, provide some necessary semantics?&lt;/li&gt;
&lt;/ol&gt;
</comment>
                            <comment id="2102851" author="judah.schvimer" created="Wed, 2 Jan 2019 02:28:53 +0000"  >&lt;p&gt;Are we in agreement then that the work here is to relax the WT invariant? &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, since this invariant is in WT, should I assign this ticket to the storage team to relax the invariant?&lt;/p&gt;</comment>
                            <comment id="2101715" author="daniel.gottlieb@10gen.com" created="Fri, 28 Dec 2018 16:27:09 +0000"  >&lt;p&gt;I&apos;ve disabled tests that depend on this ticket being resolved in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38783&quot; title=&quot;Disable tests that exercise overlapping prepared and read timestamps&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38783&quot;&gt;&lt;del&gt;SERVER-38783&lt;/del&gt;&lt;/a&gt;. This ticket (or a follow-on ticket) should re-enable the tests in &lt;tt&gt;etc/evergreen.yml&lt;/tt&gt; &lt;a href=&quot;https://github.com/mongodb/mongo/blob/65306c8e70f831e281c91200757d078d13348674/etc/evergreen.yml#L8683&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;annotated with&lt;/a&gt; &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-38499&quot; title=&quot;Preparing transaction fails and triggers invariant if chosen timestamp is not greater than WiredTiger&amp;#39;s latest active read timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-38499&quot;&gt;&lt;del&gt;SERVER-38499&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;</comment>
                            <comment id="2101417" author="daniel.gottlieb@10gen.com" created="Fri, 28 Dec 2018 04:41:57 +0000"  >&lt;blockquote&gt;
&lt;p&gt;My understanding of Judah&apos;s comments is we can keep the invariant but only for reads with ignore_prepared=false. This probably means we&apos;d have to track another timestamp for active reads with ignore_prepared=false in WT.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Ah, thanks &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;. Now I follow. At least from my testing, WT does not allow preparing behind any transactions with a read timestamp, regardless of their &lt;tt&gt;ignore_prepared&lt;/tt&gt; setting. I believe you&apos;re right; the direct way to implement a precise amount of leniency would require a separately maintained data structure.&lt;/p&gt;</comment>
                            <comment id="2101183" author="siyuan.zhou@10gen.com" created="Thu, 27 Dec 2018 19:56:05 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=daniel.gottlieb&quot; class=&quot;user-hover&quot; rel=&quot;daniel.gottlieb&quot;&gt;daniel.gottlieb&lt;/a&gt;, yes, I believe this is the issue described in this ticket, assuming &quot;Timestamp :commit 10 (creates hole)&quot; means reserving the OplogSlot and calling &lt;tt&gt;RecoveryUnit::setPrepareTimestamp()&lt;/tt&gt;.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;If the invariant is removed, I&apos;m still not sure WT would behave as required w.r.t the reader.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;My understanding of Judah&apos;s comments is we can keep the invariant but only for reads with &lt;tt&gt;ignore_prepared=false&lt;/tt&gt;. This probably means we&apos;d have to track another timestamp for active reads with &lt;tt&gt;ignore_prepared=false&lt;/tt&gt; in WT.&lt;/p&gt;</comment>
                            <comment id="2100922" author="daniel.gottlieb@10gen.com" created="Thu, 27 Dec 2018 15:00:26 +0000"  >&lt;p&gt;I understand &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tess.avitabile&quot; class=&quot;user-hover&quot; rel=&quot;tess.avitabile&quot;&gt;tess.avitabile&lt;/a&gt; are on vacation. cc &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The argument Judah gives about guarantees makes sense to me. To put those words into a concrete example, this would be an expected sequence of commands/outcomes:&lt;/p&gt;
&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Client/Preparer                                     | Oplog Writer for Prepare            | Random Writer        | Reader                                        |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|-----------------------------------------------------+-------------------------------------+----------------------+-----------------------------------------------|&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Begin Txn                                           |                                     |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Insert A                                            |                                     |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     | Begin Txn                           |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     | Timestamp :commit 10 (creates hole) |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     | Begin Txn            |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     | Timestamp :commit 20 |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     | Commit Txn           |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     |                      | Begin Txn                                     |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     |                      | Timestamp :readAt 20                          |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     |                      | &amp;lt;wait for all earlier to become visible (20)&amp;gt; |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;| Prepare 10 (invariants, but Storage should succeed) |                                     |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     | Insert oplog                        |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     | Commit Txn (fills hole)             |                      |                                               |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     |                      | &amp;lt;Proceeds&amp;gt;                                    |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;|                                                     |                                     |                      | Read A (blocks because it&apos;s prepared)         |&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=siyuan.zhou&quot; class=&quot;user-hover&quot; rel=&quot;siyuan.zhou&quot;&gt;siyuan.zhou&lt;/a&gt; can you confirm this is the scenario this ticket is describing as a problem? If the invariant is removed, I&apos;m still not sure WT would behave as required w.r.t the reader.&lt;/p&gt;</comment>
                            <comment id="2089230" author="judah.schvimer" created="Thu, 13 Dec 2018 03:26:00 +0000"  >&lt;blockquote&gt;
&lt;p&gt;we would read a version that may or may not have been correct at our time&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;For a local, available, or majority read without &lt;tt&gt;afterClusterTime&lt;/tt&gt; we don&apos;t make any guarantees about what point in time you read at. If a storage-transaction for a regular update is in flight concurrently with the read, there are no guarantees about whether or not you see that update. The same applies for prepared transactions. Non-timestamped reads that occur concurrently with a prepared transaction should safely be able to read the pre or post-image of the transaction.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;At the same time we&apos;re trying to use the fact that all reads happen at a specific time to be able to correlate shard version info. So, I&apos;m concerned that in some cases we assume snapshot semantics where everything happens at a specific timestamp, and other cases we assume that we don&apos;t actually care about snapshot semantics.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;What shard version info are you referring to? And when would we assume snapshot semantics without providing snapshot read concern, or doing everything it does to ensure proper snapshot semantics?&lt;/p&gt;

&lt;p&gt;Can we keep the invariant there only for reads with &lt;tt&gt;ignore_prepared=false&lt;/tt&gt;? This should be set correctly after &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-36382&quot; title=&quot;only snapshot, linearizable, and afterClusterTime reads should block on prepare conflicts&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-36382&quot;&gt;&lt;del&gt;SERVER-36382&lt;/del&gt;&lt;/a&gt;. &lt;/p&gt;

&lt;p&gt;The only alternative I can see to this is (from &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35798&quot; title=&quot;Writing an oplog entry for prepare should not push the lastApplied timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35798&quot;&gt;&lt;del&gt;SERVER-35798&lt;/del&gt;&lt;/a&gt;):&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;One idea we think could work well is if we were able to call prepareTransaction() with no timestamp and block all reads on the affected documents, and later after writing the oplog entry and getting a prepareTimestamp we can set the prepareTimestamp and the blocking will begin to only happen on reads after the prepareTimestamp&lt;/p&gt;&lt;/blockquote&gt;
</comment>
                            <comment id="2089089" author="geert.bosch" created="Wed, 12 Dec 2018 22:39:22 +0000"  >&lt;p&gt;It seems to me that we still need the invariant. If we read before the prepare time with a normal read operation, say at the last applied time on a secondary, we would read a version that may or may not have been correct at our time. IIUC, you&apos;re saying that in most cases we don&apos;t really care about snapshot semantics, and this is OK. At the same time we&apos;re trying to use the fact that all reads happen at a specific time to be able to correlate shard version info. So, I&apos;m concerned that in some cases we assume snapshot semantics where everything happens at a specific timestamp, and other cases we assume that we don&apos;t actually care about snapshot semantics. &lt;/p&gt;</comment>
                            <comment id="2087961" author="geert.bosch" created="Wed, 12 Dec 2018 02:38:22 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=judah.schvimer&quot; class=&quot;user-hover&quot; rel=&quot;judah.schvimer&quot;&gt;judah.schvimer&lt;/a&gt;I&apos;m not clear on all the intricacies here. Maybe we can discuss tomorrow?&lt;/p&gt;</comment>
                            <comment id="2087908" author="judah.schvimer" created="Wed, 12 Dec 2018 00:19:42 +0000"  >&lt;p&gt;This was first discussed in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35798&quot; title=&quot;Writing an oplog entry for prepare should not push the lastApplied timestamp&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35798&quot;&gt;&lt;del&gt;SERVER-35798&lt;/del&gt;&lt;/a&gt;:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;There is a period of time when a read can come in after we choose the prepareTimestamp but before we&apos;ve prepared the transaction. This will now be safe though, because any read that cares, which is any with readConcern: snapshot or readConcern: *, afterClusterTime: &amp;gt; prepareTimestamp will call waitForAllEarlierOplogWritesToBeVisible as of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-35821&quot; title=&quot;readConcern:snapshot transactions need a read timestamp &amp;lt;= WT&amp;#39;s all_committed point&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-35821&quot;&gt;&lt;del&gt;SERVER-35821&lt;/del&gt;&lt;/a&gt;, which would wait for the prepare oplog entry to be visible before trying to do the read, and at that point the transaction will already be prepared.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I think the outcome of this discussion that we didn&apos;t follow through on was relaxing the above invariant in WT. &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=geert.bosch&quot; class=&quot;user-hover&quot; rel=&quot;geert.bosch&quot;&gt;geert.bosch&lt;/a&gt;, do you agree? Should we move this to be a WT ticket to relax that invariant? &lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                        <issuelink>
            <issuekey id="665226">SERVER-38832</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="668173">SERVER-38906</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10020">
                    <name>Gantt Dependency</name>
                                            <outwardlinks description="has to be done before">
                                        <issuelink>
            <issuekey id="651748">SERVER-38569</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="717133">SERVER-40145</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="564018">SERVER-35798</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1768990">SERVER-57443</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="580523">SERVER-36382</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>25.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[5006R00001srzRjQAI]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 12 Dec 2018 00:19:42 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 2 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<s><a href='https://jira.mongodb.org/browse/SERVER-38832'>SERVER-38832</a></s>, <s><a href='https://jira.mongodb.org/browse/SERVER-38906'>SERVER-38906</a></s>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-1032</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 2 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_16465" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Linked BF Score</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>77.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>geert.bosch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>jack.mulrow@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>siyuan.zhou@mongodb.com</customfieldvalue>
            <customfieldvalue>tess.avitabile@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hufetj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu5ivb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="2607">Repl 2018-12-17</customfieldvalue>
    <customfieldvalue id="2701">Repl 2019-01-14</customfieldvalue>
    <customfieldvalue id="2709">Storage NYC 2019-01-28</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huf12v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>