<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:30:00 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-32358] Rollback can used unbounded memory</title>
                <link>https://jira.mongodb.org/browse/SERVER-32358</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;During rollback as we scan backwards in our oplog finding entries that must be rolled back, we save in memory copies of the the entire oplog entry &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v3.4/src/mongo/db/repl/rs_rollback.cpp#L180&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v3.4/src/mongo/db/repl/rs_rollback.cpp#L328&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;, if I&apos;m reading the code correctly. If there is a lot of oplog to roll back this could be a lot of memory, resulting in OOM.&lt;/p&gt;

&lt;p&gt;This code appears to have completely changed in 3.6, and I think it may not have the same problem. If so we can mark this issue fixed in 3.6 and/or dup it to the ticket that fixed the issue, using this ticket to track the issue and the fix.&lt;/p&gt;</description>
                <environment></environment>
        <key id="472916">SERVER-32358</key>
            <summary>Rollback can used unbounded memory</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="backlog-server-repl">Backlog - Replication Team</assignee>
                                    <reporter username="bruce.lucas@mongodb.com">Bruce Lucas</reporter>
                        <labels>
                    </labels>
                <created>Fri, 15 Dec 2017 14:52:49 +0000</created>
                <updated>Tue, 6 Dec 2022 03:43:43 +0000</updated>
                            <resolved>Fri, 15 Dec 2017 21:47:42 +0000</resolved>
                                    <version>3.4.10</version>
                                                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="1762132" author="bruce.lucas@10gen.com" created="Sun, 31 Dec 2017 15:08:58 +0000"  >&lt;p&gt;The issue reported in this ticket is not related to memory usage by WT, but rather is memory usage by the rollback algorithm outside the WT cache. Cache pressure related to lagging majority commit point could cause performance issues, but should not cause excessive memory usage and OOM, which is the issue of concern here. &lt;/p&gt;</comment>
                            <comment id="1762129" author="schwerin" created="Sun, 31 Dec 2017 14:56:44 +0000"  >&lt;p&gt;The WT resource usage increases as the node&apos;s local commit point gets far ahead of the majority commit point in 3.6 as well as 3.8.&lt;/p&gt;

&lt;p&gt;Recover to timestamp is indeed a 3.8 feature.&lt;/p&gt;</comment>
                            <comment id="1753528" author="spencer" created="Fri, 15 Dec 2017 22:12:14 +0000"  >&lt;p&gt;In 3.8, the further ahead of the replication majority commit point a node gets, the more WiredTiger cache pressure there will be, so before you get a large rollback you&apos;ll likely already be having problems.  Outside of that, however, there shouldn&apos;t be any part of the new rollback that scales memory usage relative to the size of the rollback.&lt;/p&gt;</comment>
                            <comment id="1753525" author="bruce.lucas@10gen.com" created="Fri, 15 Dec 2017 22:07:10 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=spencer&quot; class=&quot;user-hover&quot; rel=&quot;spencer&quot;&gt;spencer&lt;/a&gt; how much better, given Judah&apos;s &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-32358?focusedCommentId=1753037&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1753037&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;comment&lt;/a&gt; above? Is there still a possibility of OOM? If so shouldn&apos;t we bound the amount of memory used and abort rather than OOM in order to provide a useful error message?&lt;/p&gt;</comment>
                            <comment id="1753504" author="spencer" created="Fri, 15 Dec 2017 21:47:42 +0000"  >&lt;p&gt;I don&apos;t think we&apos;re going to do anything here for our existing rollback algorithms.  The situation should be better in the new Recover to Timestamp based rollback algorithm in 3.6.&lt;/p&gt;</comment>
                            <comment id="1753327" author="judah.schvimer" created="Fri, 15 Dec 2017 19:32:04 +0000"  >&lt;p&gt;We do constrain the amount of data &lt;a href=&quot;https://github.com/mongodb/mongo/blob/b1ae796faf5f262cd658f59a59d18f8c3eb0d4ee/src/mongo/db/repl/rs_rollback.cpp#L1033-L1038&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. I guess that constraint does not include the documents getting rolled back, just the refetched documents.&lt;/p&gt;</comment>
                            <comment id="1753084" author="bruce.lucas@10gen.com" created="Fri, 15 Dec 2017 15:59:21 +0000"  >&lt;p&gt;Since in general we place a couple restrictions on the &quot;amount&quot; of rollback we do, it might be ok to track how much memory we are using and abort the rollback when we use &quot;too much&quot; (could be hard-coded limit, could be computed from allocated vs physical memory). This would be better than OOM because it would make the cause more easily diagnosable and could point to the workaround of doing an initial sync.&lt;/p&gt;</comment>
                            <comment id="1753037" author="judah.schvimer" created="Fri, 15 Dec 2017 15:12:11 +0000"  >&lt;p&gt;I believe this is still a problem &lt;a href=&quot;https://github.com/mongodb/mongo/blob/v3.6/src/mongo/db/repl/rs_rollback.cpp#L570&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. It would be fairly straightforward to persist these documents to disk. We probably could also only save the _id and the sync source&apos;s copy of the document once we refetch it. These were all considered out of scope for the &quot;Safe Rollback For Non-WT Storage Engine&quot; project. I don&apos;t think this ticket has a duplicate, so I&apos;ll leave it open for now.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="275717">SERVER-23392</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25128"><![CDATA[Replication]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 15 Dec 2017 15:12:11 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 6 weeks, 3 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 6 weeks, 3 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-repl</customfieldvalue>
            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>judah.schvimer@mongodb.com</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htmco7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htdxsf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|htlysn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>