<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:48:55 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-18844] Reacquire the snapshot after commit/abort</title>
                <link>https://jira.mongodb.org/browse/SERVER-18844</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;If I understand correctly, Mongo&apos;s contract is to always reacquire the snapshot (i.e. do saveState + restoreState) after each commit/abort.&lt;/p&gt;

&lt;p&gt;There is (at least) one place in the code where this is not true. To demonstrate the issue I created a patch here: &lt;a href=&quot;https://github.com/mongodb-partners/mongo/commits/v3.0-failure&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb-partners/mongo/commits/v3.0-failure&lt;/a&gt; (on top of the current v3.0 branch)&lt;/p&gt;

&lt;p&gt;The patch keeps track of live iterators and makes sure that they&apos;re not reused after commit. Here&apos;s the stack trace of invalidation failure: &lt;a href=&quot;https://gist.github.com/igorcanadi/c15eb094583054a0918a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://gist.github.com/igorcanadi/c15eb094583054a0918a&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This is where the commit happens: &lt;a href=&quot;https://github.com/mongodb/mongo/blob/master/src/mongo/db/catalog/index_create.cpp#L272&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/master/src/mongo/db/catalog/index_create.cpp#L272&lt;/a&gt;, while `exec` plan executor is not yielding.&lt;/p&gt;

&lt;p&gt;This issue is causing background index build concurrency problems for RocksDB, as evidenced by &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18744&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/SERVER-18744&lt;/a&gt;. We&apos;ve encountered this in production.&lt;/p&gt;

&lt;p&gt;It would also be good to add some invariants in the code to make sure this contract is respected, since it&apos;s a bit tricky behavior.&lt;/p&gt;

&lt;p&gt;Let me know if I&apos;m misunderstanding anything.&lt;/p&gt;</description>
                <environment></environment>
        <key id="208771">SERVER-18844</key>
            <summary>Reacquire the snapshot after commit/abort</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="2" iconUrl="https://jira.mongodb.org/images/icons/priorities/critical.svg">Critical - P2</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="kyle.suarez@mongodb.com">Kyle Suarez</assignee>
                                    <reporter username="igor">Igor Canadi</reporter>
                        <labels>
                            <label>code-only</label>
                    </labels>
                <created>Fri, 5 Jun 2015 18:21:46 +0000</created>
                <updated>Thu, 2 Aug 2018 21:22:54 +0000</updated>
                            <resolved>Thu, 21 Apr 2016 19:04:10 +0000</resolved>
                                    <version>3.3.4</version>
                                    <fixVersion>3.2.6</fixVersion>
                    <fixVersion>3.3.5</fixVersion>
                                    <component>Index Maintenance</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="1242934" author="xgen-internal-githook" created="Thu, 21 Apr 2016 19:12:59 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;ksuarz&apos;, u&apos;name&apos;: u&apos;Kyle Suarez&apos;, u&apos;email&apos;: u&apos;ksuarz@gmail.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22970&quot; title=&quot;Background index build produces an index with mismatched index keys and documents&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22970&quot;&gt;&lt;del&gt;SERVER-22970&lt;/del&gt;&lt;/a&gt; fix update race with background index build&lt;/p&gt;

&lt;p&gt;There are two distinct fixes that must be done together:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23807&quot; title=&quot;Updates should always throw WriteConflictException on unindexing&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23807&quot;&gt;&lt;del&gt;SERVER-23807&lt;/del&gt;&lt;/a&gt;: update should throw write conflict on unindex&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18844&quot; title=&quot;Reacquire the snapshot after commit/abort&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18844&quot;&gt;&lt;del&gt;SERVER-18844&lt;/del&gt;&lt;/a&gt;: background index builds should reacquire the snapshot to&lt;br/&gt;
                properly detect write conflicts with concurrent updates&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Branch: v3.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/be81cc9e83d09d4dc206c59656fc7e51c3e4fc12&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/be81cc9e83d09d4dc206c59656fc7e51c3e4fc12&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1242909" author="xgen-internal-githook" created="Thu, 21 Apr 2016 19:01:25 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;ksuarz&apos;, u&apos;name&apos;: u&apos;Kyle Suarez&apos;, u&apos;email&apos;: u&apos;ksuarz@gmail.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-22970&quot; title=&quot;Background index build produces an index with mismatched index keys and documents&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-22970&quot;&gt;&lt;del&gt;SERVER-22970&lt;/del&gt;&lt;/a&gt; fix update race with background index build&lt;/p&gt;

&lt;p&gt;There are two distinct fixes that must be done together:&lt;/p&gt;

&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-23807&quot; title=&quot;Updates should always throw WriteConflictException on unindexing&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-23807&quot;&gt;&lt;del&gt;SERVER-23807&lt;/del&gt;&lt;/a&gt;: update should throw write conflict on unindex&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18844&quot; title=&quot;Reacquire the snapshot after commit/abort&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18844&quot;&gt;&lt;del&gt;SERVER-18844&lt;/del&gt;&lt;/a&gt;: background index builds should reacquire the snapshot to&lt;br/&gt;
                properly detect write conflicts with concurrent updates&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/f27b0cac4869fa506c6ed6f0dfc885b9edcd765a&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/f27b0cac4869fa506c6ed6f0dfc885b9edcd765a&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1068153" author="xgen-internal-githook" created="Thu, 22 Oct 2015 18:27:17 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;igorcanadi&apos;, u&apos;name&apos;: u&apos;Igor Canadi&apos;, u&apos;email&apos;: u&apos;icanadi@fb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20650&quot; title=&quot;Backport MongoRocks changes to 3.0&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20650&quot;&gt;&lt;del&gt;SERVER-20650&lt;/del&gt;&lt;/a&gt; backport recent changes from MongoRocks&lt;/p&gt;

&lt;p&gt;This patch includes the following changes:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Don&apos;t lazily initialize _writeBatch&lt;/li&gt;
	&lt;li&gt;Abandon snapshots in commitAndRestart()&lt;/li&gt;
	&lt;li&gt;Use only 33% of RAM for MongoRocks block cache&lt;/li&gt;
	&lt;li&gt;Decrease block cache shard count to 2^6 from 2^7&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18844&quot; title=&quot;Reacquire the snapshot after commit/abort&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18844&quot;&gt;&lt;del&gt;SERVER-18844&lt;/del&gt;&lt;/a&gt; Refresh iterators on commit()&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-17062&quot; title=&quot;Make query execution robust to WriteConflictException&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-17062&quot;&gt;&lt;del&gt;SERVER-17062&lt;/del&gt;&lt;/a&gt; Allow cursors over unique indexes with duplicates&lt;/li&gt;
	&lt;li&gt;Recompute dataSize and numRecords if numbers don&apos;t make sense&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Signed-off-by: Ramon Fernandez &amp;lt;ramon@mongodb.com&amp;gt;&lt;br/&gt;
Branch: v3.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/2a4e6cd054ad8b1d3c54ca73fc6d852116eb89c5&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/2a4e6cd054ad8b1d3c54ca73fc6d852116eb89c5&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="987297" author="dynamike" created="Thu, 23 Jul 2015 18:23:15 +0000"  >&lt;p&gt;any feedback here?&lt;/p&gt;</comment>
                            <comment id="969079" author="dynamike" created="Fri, 17 Jul 2015 19:41:13 +0000"  >&lt;p&gt;What I&apos;ve been able to find so far is operations on the database building the background index are stuck waiting on a lock. Looking at db.currentOp() doesn&apos;t show anything holding an exclusive lock. All other operations on that database have waitingForLock:true. So something seems to be lying about holding a database lock and I can&apos;t tell where. &lt;/p&gt;

&lt;p&gt;Also, lock contention isn&apos;t immediately triggered once the background index build starts. It only happens after some arbitrary period of time during the index build. I&apos;m trying to see if a specific operation triggers this, but haven&apos;t found it yet. &lt;/p&gt;</comment>
                            <comment id="968672" author="igor" created="Fri, 17 Jul 2015 12:17:53 +0000"  >&lt;p&gt;We have been running my temporary fix &lt;span class=&quot;error&quot;&gt;&amp;#91;1&amp;#93;&lt;/span&gt; in production, but it&apos;s causing some performance issues. Specifically, bunch of getmore() calls get stuck when we build an index (they take 5 seconds, while they used to take 9ms). Once the index build is done, latencies drop back to normal. Do you have any idea what might cause this? I&apos;m also interested in tweaks that Mathias was thinking about. Hopefully the tweaks will solve our problems &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;&lt;span class=&quot;error&quot;&gt;&amp;#91;1&amp;#93;&lt;/span&gt; &lt;a href=&quot;https://github.com/mongodb-partners/mongo/commit/434888864ef4415921a4f2ad2a184447f394eaa7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb-partners/mongo/commit/434888864ef4415921a4f2ad2a184447f394eaa7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="953533" author="martin.bligh" created="Mon, 29 Jun 2015 19:18:20 +0000"  >&lt;p&gt;Mathias had a quick look at this and was going to make some small tweaks before applying. &lt;/p&gt;</comment>
                            <comment id="948357" author="igor" created="Tue, 23 Jun 2015 17:23:39 +0000"  >&lt;p&gt;Hi Martin! Are there any updates for this task? I see that it&apos;s marked as &quot;needs triage&quot;. Do you think the fix might make it into 3.0.5 version?&lt;/p&gt;

&lt;p&gt;I would also really appreciate if you could look at my fix (it&apos;s only two lines) and give your opinion on the correctness.&lt;/p&gt;</comment>
                            <comment id="934306" author="igor" created="Mon, 8 Jun 2015 18:56:12 +0000"  >&lt;p&gt;Here&apos;s my temporary fix: &lt;a href=&quot;https://github.com/mongodb-partners/mongo/commit/434888864ef4415921a4f2ad2a184447f394eaa7&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb-partners/mongo/commit/434888864ef4415921a4f2ad2a184447f394eaa7&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="932954" author="igor" created="Fri, 5 Jun 2015 18:23:32 +0000"  >&lt;p&gt;s/not committed/not yielded&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="245210">SERVER-22062</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="280825">SERVER-23807</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="178346">SERVER-16816</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_12451" key="com.atlassian.jira.plugin.system.customfieldtypes:multiversion">
                        <customfieldname>Backport Completed</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="16812">3.2.6</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 29 Jun 2015 19:18:20 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 42 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>greg.mckeon@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 42 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>igor</customfieldvalue>
            <customfieldvalue>kyle.suarez@mongodb.com</customfieldvalue>
            <customfieldvalue>martin.bligh</customfieldvalue>
            <customfieldvalue>dynamike</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrixp3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrdjk7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="912">Integration 10 (02/22/16)</customfieldvalue>
    <customfieldvalue id="994">Integration 13 (04/22/16)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11861" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>User Summary</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="11856"><![CDATA[Not Needed]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfypr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>