<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:25:32 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-73757] Calling storageSize() on ephemeral temporary record stores returns 0</title>
                <link>https://jira.mongodb.org/browse/SERVER-73757</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;When we create a temporary record store, the field &lt;a href=&quot;http://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp/#L1811-L1812&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_tracksSizeAdjustment is set to false&lt;/a&gt;.&#160;&lt;/p&gt;

&lt;p&gt;On inserting records, the method &lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp/#L1400-L1401&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;_changeNumRecordsAndDataSize&lt;/a&gt;&#160;is called to add the statistics. The method &lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp/#L1984-L1986&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;returns without modifying the _sizeInfo when _tracksSizeAdjustment is set to false&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;When calling storageSize() to get the size, if storage engine is ephemeral, dataSize() is called which uses _sizeInfo.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2257203">SERVER-73757</key>
            <summary>Calling storageSize() on ephemeral temporary record stores returns 0</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10038" iconUrl="https://jira.mongodb.org/images/icons/subtask.gif" description="">Backlog</status>
                    <statusCategory id="2" key="new" colorName="default"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="adi.agrawal@mongodb.com">Adi Agrawal</reporter>
                        <labels>
                    </labels>
                <created>Wed, 8 Feb 2023 14:56:09 +0000</created>
                <updated>Thu, 23 Feb 2023 21:22:26 +0000</updated>
                                                                                                <votes>0</votes>
                                    <watches>4</watches>
                                                                                                                <comments>
                            <comment id="5185635" author="louis.williams" created="Fri, 10 Feb 2023 08:22:29 +0000"  >&lt;blockquote&gt;&lt;p&gt;When you say this are you referring specifically to the SizeStorer? The observation we made is that the $group spill stats seem to be incorrect only on inMemory. The reason is presumably as Adi observes above, which is that the storageSize() method uses WiredTigerUtil::getStatisticsValue() for non-ephemeral record stores.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;Yeah, I think we&apos;re in agreement on the problem. We explicitly don&apos;t update the SizeStorer for these temporary record stores. The getStatisticsValue() call would not return useful information, so on inMemory, we return whatever is in the size storer. But in this case, it always returns zero.&lt;/p&gt;</comment>
                            <comment id="5184223" author="david.storch" created="Thu, 9 Feb 2023 19:47:36 +0000"  >&lt;blockquote&gt;&lt;p&gt;&#160;&lt;br/&gt;
What is the priority of this?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;The impact that we&apos;re observing is that the execution stats exposed by SBE&apos;s $group implementation in explain and serverStatus are wrong when spilling occurs and the in-memory storage engine is in use. So it&apos;s a diagnostics problem, which might reduce its priority. I think the SE team is in a better position to evaluate how we want to prioritize fixes for inMemory specifically against other work. (I don&apos;t know the adoption of inMemory amongst our user base, for example.) I would add that if we&apos;re going to work on any improvement in this area, I would argue it would be more important for us to spend time on related ticket &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-73717&quot; title=&quot;When using in-memory storage engine spill to disk doesn&amp;#39;t always work&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-73717&quot;&gt;&lt;del&gt;SERVER-73717&lt;/del&gt;&lt;/a&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;No, no TemporaryRecordStores track size information.&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;When you say this are you referring specifically to the &lt;tt&gt;SizeStorer&lt;/tt&gt;? The observation we made is that the $group spill stats seem to be incorrect only on inMemory. The reason is presumably as Adi observes above, which is that the &lt;tt&gt;storageSize()&lt;/tt&gt; method uses &lt;a href=&quot;https://github.com/10gen/mongo/blob/f139c94b7ae8961f5b18df0e95e9708b3710929b/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp#L1042-L1045&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&lt;tt&gt;WiredTigerUtil::getStatisticsValue()&lt;/tt&gt;&lt;/a&gt; for non-ephemeral record stores.&lt;br/&gt;
&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="5181754" author="louis.williams" created="Thu, 9 Feb 2023 08:51:32 +0000"  >&lt;blockquote&gt;&lt;p&gt;it&apos;s only true for ephemeral temporary record stores. Is that correct?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;No, no TemporaryRecordStores track size information.&lt;/p&gt;</comment>
                            <comment id="5181728" author="louis.williams" created="Thu, 9 Feb 2023 08:42:50 +0000"  >&lt;p&gt;StorEx can take this ticket. If I remember why we made this decision originally, it was in part because the SizeStorer has a slow memory leak: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-33494&quot; title=&quot;WT SizeStorer never deletes old entries&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-33494&quot;&gt;&lt;del&gt;SERVER-33494&lt;/del&gt;&lt;/a&gt;. We don&apos;t delete size information for record stores once they have been dropped. So as part of this fix, we may need to evaluate the consequences of creating more stale entires in the SizeStorer.&lt;/p&gt;

&lt;p&gt;What is the priority of this?&lt;/p&gt;</comment>
                            <comment id="5180071" author="JIRAUSER1260499" created="Wed, 8 Feb 2023 20:02:29 +0000"  >&lt;p&gt;Thanks for catching this. It should apply only to ephemeral record stores.&lt;/p&gt;

&lt;p&gt;This is because of how storageSize() is implemented. If the storage engine is in memory, then &lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_record_store.cpp/#L1038-L1040&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;storageSize()&lt;/a&gt; calls dataSize() which checks the _sizeInfo. However, if the storage engine is not in memory, the size is obtained from&lt;br/&gt;
&lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp/#L500-L509&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;WiredTigerUtil::getStatisticsValue&lt;/a&gt;&#160;&lt;br/&gt;
&#160;&lt;/p&gt;</comment>
                            <comment id="5179886" author="david.storch" created="Wed, 8 Feb 2023 19:18:15 +0000"  >&lt;p&gt;If it applies to non-ephemeral record stores, then how come the problem we observed around the $group spilling stats in &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-73311&quot; title=&quot;Report group spill stats in serverStatus for SBE&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-73311&quot;&gt;&lt;del&gt;SERVER-73311&lt;/del&gt;&lt;/a&gt; only manifested for the inMemory storage engine?&lt;/p&gt;</comment>
                            <comment id="5178880" author="JIRAUSER1260499" created="Wed, 8 Feb 2023 15:29:56 +0000"  >&lt;p&gt;&lt;del&gt;I think it applies to all temporary record stores. The field &lt;a href=&quot;#L1811-L1812&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;_tracksSizeAdjustment is set to false&lt;/a&gt;&#160;regardless of if the record store is ephemeral. (&lt;a href=&quot;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp/#L1811-L1812&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/10gen/mongo/blob/master/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp/#L1811-L1812&lt;/a&gt;)&lt;/del&gt;&lt;/p&gt;</comment>
                            <comment id="5178740" author="david.storch" created="Wed, 8 Feb 2023 15:06:39 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=adi.agrawal%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;adi.agrawal@mongodb.com&quot;&gt;adi.agrawal@mongodb.com&lt;/a&gt; to clarify, this isn&apos;t true for &lt;em&gt;all&lt;/em&gt; temporary record stores, it&apos;s only true for ephemeral temporary record stores. Is that correct?&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=louis.williams%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;louis.williams@mongodb.com&quot;&gt;louis.williams@mongodb.com&lt;/a&gt; is this something that your team could schedule a fix for, assuming you still believe that it is incorrect behavior?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                        <issuelink>
            <issuekey id="2273897">SERVER-74335</issuekey>
        </issuelink>
                            </outwardlinks>
                                                                <inwardlinks description="is related to">
                                        <issuelink>
            <issuekey id="2255090">SERVER-73717</issuekey>
        </issuelink>
            <issuelink>
            <issuekey id="2243423">SERVER-73311</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>8.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 8 Feb 2023 15:06:39 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        51 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            51 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>adi.agrawal@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>david.storch@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1uqfb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1d6q8:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10750" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Steps To Reproduce</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>&lt;ul&gt;
	&lt;li&gt;create a temporary record store&lt;/li&gt;
	&lt;li&gt;insert some records&lt;/li&gt;
	&lt;li&gt;call storageSize()&lt;/li&gt;
&lt;/ul&gt;
</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1uckn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>