<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:09:06 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DOCS-13943] Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries</title>
                <link>https://jira.mongodb.org/browse/DOCS-13943</link>
                <project id="10380" key="DOCS">Documentation</project>
                    <description>&lt;h2&gt;&lt;a name=&quot;Description&quot;&gt;&lt;/a&gt;Description&lt;/h2&gt;

    &lt;div class=&quot;panel&quot; style=&quot;background-color: #c2d2c2;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelHeader&quot; style=&quot;border-bottom-width: 1px;background-color: #239eb0;&quot;&gt;&lt;b&gt;Downstream Change Summary&lt;/b&gt;&lt;/div&gt;&lt;div class=&quot;panelContent&quot; style=&quot;background-color: #c2d2c2;&quot;&gt;
&lt;p&gt;    There is a new setParameter &apos;internalQueryCacheMaxSizeBytesBeforeStripDebugInfo&apos;. Its default value is 0.5 GB. The operator can increase the value of the parameter to allow more debug information to be stored in the plan cache, or reduce its value to save memory. By setting it to zero, no debug information will be stored alongside new plan cache entries.&lt;/p&gt;

&lt;p&gt;When the debug info for a plan cache entry has been stripped, $planCacheStats will return an entry that has the fields &quot;queryHash&quot;, &quot;planCacheKey&quot;, &quot;isActive&quot;, &quot;works&quot;, &quot;timeOfCreation&quot;, &quot;indexFilterSet&quot;, and &quot;estimatedSizeBytes&quot;, but it will be missing the fields &quot;createdFromQuery&quot;, &quot;cachedPlan&quot;, &quot;creationExecStats&quot;, and &quot;candidatePlanScores&quot;. Note that &quot;estimatedSizeBytes&quot; is a new field which displays the estimated size in bytes of each plan cache entry.&lt;/p&gt;

&lt;p&gt;For documentation purposes, I&apos;m not sure that we should document &apos;internalQueryCacheMaxSizeBytesBeforeStripDebugInfo&apos;, since hopefully the default will be appropriate for most applications. However, we should document the addition of the &quot;estimatedSizeBytes&quot; field to the $planCacheStats output.&lt;/p&gt;
&lt;/div&gt;&lt;/div&gt;
&lt;h2&gt;&lt;a name=&quot;DescriptionofLinkedTicket&quot;&gt;&lt;/a&gt;Description of Linked Ticket&lt;/h2&gt;
&lt;p&gt;    For some workloads, the per-collection query plan caches can consume too much memory. &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-34886&quot; title=&quot;Plan cache size is bounded by number of entries rather than memory footprint&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-34886&quot;&gt;SERVER-34886&lt;/a&gt; and &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40360&quot; title=&quot;Process-global plan cache&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-40360&quot;&gt;SERVER-40360&lt;/a&gt; together describe an architectural solution to this problem: turn the per-collection plan caches into a process-global plan cache, and bound the size of this new global cache in bytes rather than number of entries.&lt;/p&gt;

&lt;p&gt;However, we could also reduce the memory footprint of the plan cache by auditing the debug information it holds in memory. We have observed in the field that this problem is most severe when the user runs large queries; this can result in the information held in the plan cache for introspection taking up a lot of memory. Perhaps this debug information can be truncated if it exceeds some threshold, in order to reduce the amount of debug information present in exchange for avoidance of excessive memory consumption.&lt;/p&gt;

&lt;p&gt;The suspected worst offenders are:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/6b8b02e8d72c055c6f686a781e53cde9384461b3/src/mongo/db/exec/index_scan.cpp#L273&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;A serialized representation of the index bounds&lt;/a&gt; for an IXSCAN stage. We&apos;ve seen this for queries with &lt;tt&gt;$in&lt;/tt&gt; predicates that have hundreds of thousands of elements. If the &lt;tt&gt;$in&lt;/tt&gt; is converted into index bounds, the serialized bounds will consist of hundreds of thousands of point intervals.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/6b8b02e8d72c055c6f686a781e53cde9384461b3/src/mongo/db/query/plan_cache.h#L262&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;A copy of the query predicate&lt;/a&gt;. This can approach 16MB on its own per plan entry if the user runs large queries.&lt;/li&gt;
	&lt;li&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/6b8b02e8d72c055c6f686a781e53cde9384461b3/src/mongo/db/exec/fetch.cpp#L192&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;Another copy of the a potentially large predicate&lt;/a&gt;, held inside the stats for a FETCH stage.&lt;/li&gt;
&lt;/ul&gt;



&lt;h2&gt;&lt;a name=&quot;Scopeofchanges&quot;&gt;&lt;/a&gt;Scope of changes&lt;/h2&gt;

&lt;h2&gt;&lt;a name=&quot;ImpacttoOtherDocs&quot;&gt;&lt;/a&gt;Impact to Other Docs&lt;/h2&gt;

&lt;h2&gt;&lt;a name=&quot;MVP%28WorkandDate%29&quot;&gt;&lt;/a&gt;MVP (Work and Date)&lt;/h2&gt;

&lt;h2&gt;&lt;a name=&quot;Resources%28ScopeorDesignDocs%2CInvision%2Cetc.%29&quot;&gt;&lt;/a&gt;Resources (Scope or Design Docs, Invision, etc.)&lt;/h2&gt;
</description>
                <environment></environment>
        <key id="1522620">DOCS-13943</key>
            <summary>Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="naomi.pentrel@mongodb.com">Naomi Pentrel</assignee>
                                    <reporter username="backlog-server-pm">Backlog - Core Eng Program Management Team</reporter>
                        <labels>
                    </labels>
                <created>Thu, 22 Oct 2020 16:05:59 +0000</created>
                <updated>Mon, 13 Nov 2023 18:02:07 +0000</updated>
                            <resolved>Mon, 22 Mar 2021 17:52:22 +0000</resolved>
                                                    <fixVersion>4.9.0</fixVersion>
                    <fixVersion>4.2.12</fixVersion>
                    <fixVersion>4.4.3</fixVersion>
                    <fixVersion>4.0.23</fixVersion>
                    <fixVersion>3.6.23</fixVersion>
                    <fixVersion>Server_Docs_20231030</fixVersion>
                    <fixVersion>Server_Docs_20231106</fixVersion>
                    <fixVersion>Server_Docs_20231105</fixVersion>
                    <fixVersion>Server_Docs_20231113</fixVersion>
                                    <component>manual</component>
                    <component>Server</component>
                        <due></due>
                            <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="3708319" author="xgen-internal-githook" created="Thu, 8 Apr 2021 17:00:57 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Naomi Pentrel&apos;, &apos;email&apos;: &apos;5212232+npentrel@users.noreply.github.com&apos;, &apos;username&apos;: &apos;npentrel&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13943&quot; title=&quot;Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13943&quot;&gt;&lt;del&gt;DOCS-13943&lt;/del&gt;&lt;/a&gt; Add plan cache limit and stats&lt;br/&gt;
Branch: v3.6&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/0c38fcb1187275ad85aef74d282c3320458c2b76&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/0c38fcb1187275ad85aef74d282c3320458c2b76&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3705246" author="xgen-internal-githook" created="Wed, 7 Apr 2021 13:47:01 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Naomi Pentrel&apos;, &apos;email&apos;: &apos;5212232+npentrel@users.noreply.github.com&apos;, &apos;username&apos;: &apos;npentrel&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13943&quot; title=&quot;Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13943&quot;&gt;&lt;del&gt;DOCS-13943&lt;/del&gt;&lt;/a&gt; Add plan cache limit and stats&lt;br/&gt;
Branch: v4.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/a703c2afbfa570f71165660af468cb318c85015d&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/a703c2afbfa570f71165660af468cb318c85015d&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3684169" author="xgen-internal-githook" created="Thu, 25 Mar 2021 16:52:47 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Naomi Pentrel&apos;, &apos;email&apos;: &apos;5212232+npentrel@users.noreply.github.com&apos;, &apos;username&apos;: &apos;npentrel&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13943&quot; title=&quot;Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13943&quot;&gt;&lt;del&gt;DOCS-13943&lt;/del&gt;&lt;/a&gt; Add plan cache limit and stats&lt;br/&gt;
Branch: v4.2&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/cac9ed5382b3fb68f4980fecff48866c407ad814&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/cac9ed5382b3fb68f4980fecff48866c407ad814&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3679783" author="xgen-internal-githook" created="Tue, 23 Mar 2021 17:21:59 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Naomi Pentrel&apos;, &apos;email&apos;: &apos;5212232+npentrel@users.noreply.github.com&apos;, &apos;username&apos;: &apos;npentrel&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13943&quot; title=&quot;Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13943&quot;&gt;&lt;del&gt;DOCS-13943&lt;/del&gt;&lt;/a&gt; Add plan cache limit and stats&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/580ecd1af63e6301c50b0f9e9e934fe57bd64217&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/580ecd1af63e6301c50b0f9e9e934fe57bd64217&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3679778" author="xgen-internal-githook" created="Tue, 23 Mar 2021 17:21:06 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;name&apos;: &apos;Naomi Pentrel&apos;, &apos;email&apos;: &apos;5212232+npentrel@users.noreply.github.com&apos;, &apos;username&apos;: &apos;npentrel&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/DOCS-13943&quot; title=&quot;Investigate changes in SERVER-40361: Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;DOCS-13943&quot;&gt;&lt;del&gt;DOCS-13943&lt;/del&gt;&lt;/a&gt; Add plan cache limit and stats&lt;br/&gt;
Branch: v5.0&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/docs/commit/6f93ed1ad3911a779436eb059a77b37fcb4dc554&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/docs/commit/6f93ed1ad3911a779436eb059a77b37fcb4dc554&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="3549879" author="david.storch" created="Tue, 5 Jan 2021 22:52:50 +0000"  >&lt;p&gt;See my &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40361?focusedCommentId=3549877&amp;amp;page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-3549877&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;comment here&lt;/a&gt; in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-40361&quot; title=&quot;Reduce memory footprint of plan cache entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-40361&quot;&gt;&lt;del&gt;SERVER-40361&lt;/del&gt;&lt;/a&gt; for a note about how 4.2 version of this change differs from the 4.4+ version. Namely, in 4.2 there are consequences for the &lt;a href=&quot;https://docs.mongodb.com/v4.2/reference/command/planCacheListQueryShapes/#dbcmd.planCacheListQueryShapes&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;planCacheListQueryShapes&lt;/tt&gt;&lt;/a&gt; and &lt;a href=&quot;https://docs.mongodb.com/v4.2/reference/command/planCacheListPlans/#dbcmd.planCacheListPlans&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;planCacheListPlans&lt;/tt&gt;&lt;/a&gt; commands. In 4.4, in contrast, these commands do not exist, since they were deprecated in 4.2 and removed in 4.4.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                        <issuelink>
            <issuekey id="1546806">DOCS-14004</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1581481">DOCS-14080</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1604817">DOCS-14166</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="1606027">DOCS-14172</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                            <outwardlinks description="documents">
                                        <issuelink>
            <issuekey id="724151">SERVER-40361</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>6.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 5 Jan 2021 22:52:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 43 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>DOCSP-9747</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>emet.ozar@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 43 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-pm</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>naomi.pentrel@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hye2xb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hy0a73:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="4817">ServerDocs2020: Mar9 - Mar16</customfieldvalue>
    <customfieldvalue id="4842">ServerDocs2020: Mar16 - Mar23</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hydp6n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>