<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:11:38 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-26277] allow to have compressed data in RAM</title>
                <link>https://jira.mongodb.org/browse/SERVER-26277</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Current situation: Having compression enabled - data size = 5 GB, storage size = 1GB&lt;br/&gt;
But when data is loaded to RAM, it consumes 5 GB in RAM.&lt;/p&gt;

&lt;p&gt;Proposal:&lt;br/&gt;
What if the data /working set/ in RAM would be compressed, same as on disk ? So working set in RAM would be 1 GB and not 5GB&lt;/p&gt;

&lt;p&gt;Considering a lot of CPUs, the online decompression overhead would be minor.&lt;/p&gt;

&lt;p&gt;This would not be of course suitable for super-fast data, but majority of the apps in my company is midsize,low-speed apps. With memory-compression switch we could avoid many situations when working set is above cache and pages must be exchanged with disk and locking the db&lt;/p&gt;
</description>
                <environment></environment>
        <key id="318394">SERVER-26277</key>
            <summary>allow to have compressed data in RAM</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="5" iconUrl="https://jira.mongodb.org/images/icons/priorities/trivial.svg">Trivial - P5</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="marian.badinka@dhl.com">marian badinka</reporter>
                        <labels>
                    </labels>
                <created>Fri, 23 Sep 2016 12:52:33 +0000</created>
                <updated>Tue, 6 Dec 2022 04:16:03 +0000</updated>
                            <resolved>Fri, 30 Sep 2016 05:08:57 +0000</resolved>
                                                                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>7</watches>
                                                                                                                <comments>
                            <comment id="1397287" author="alexander.gorrod" created="Fri, 30 Sep 2016 01:41:38 +0000"  >&lt;blockquote&gt;&lt;p&gt;We would set up WT cache for &quot;modest size&quot; 1-2 GB only. So the imported data would be actively flushed from cache to disk. But what about queries ? They will most probably will not have space in RAM and every query will trigger the load of pages from disk to cache. ? Or the rest of RAM 6-7 GB would be also used to keep the working set in RAM ?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I believe your question here is whether  reads of recently written data come from filesystem cache, or need to read from disk. My understanding is that they will be read from filesystem cache, but you should definitely test your particular configuration.&lt;/p&gt;

&lt;blockquote&gt;&lt;p&gt;would you consider introduce feature to lock-down the specified collection data/&quot;query result&quot; in RAM?. F.e. &quot;login_name&quot; or &quot;session&quot; collection would be nice to have always in RAM despite there will be heavy write/read request causing to empty/exchange the whole WT-Cache ?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;We do have an open feature request to be able to give certain tables priority in cache: &lt;a href=&quot;https://jira.mongodb.org/browse/WT-2452&quot; class=&quot;external-link&quot; rel=&quot;nofollow&quot;&gt;https://jira.mongodb.org/browse/WT-2452&lt;/a&gt;&lt;br/&gt;
At the moment we rely on the cache eviction least-recently-used algorithm to choose the best pages without hints. Once the functionality is added to WiredTiger it will need to be exposed via MongoDB as well. It&apos;s on our todo list, and I recommend you add a comment stating your use case in that ticket, to increase the likelihood that the work gets scheduled.&lt;/p&gt;</comment>
                            <comment id="1396497" author="marian.badinka@dhl.com" created="Thu, 29 Sep 2016 12:06:27 +0000"  >&lt;p&gt;Alex,&lt;/p&gt;

&lt;p&gt;Thanks a lot for your update.&lt;/p&gt;

&lt;p&gt;What about another case. RAM = f.e. 8 GB&lt;/p&gt;

&lt;p&gt;We have application requiring heavy write and middle read.&lt;/p&gt;

&lt;p&gt;We would set up WT cache for &quot;modest size&quot; 1-2 GB only. So the imported data would be actively flushed from cache to disk. But what about queries ? They will most probably will not have space in RAM and every query will trigger the load of pages from disk to cache. ? Or the rest of RAM 6-7 GB would be also used to keep the working set in RAM ?&lt;/p&gt;

&lt;p&gt;Another related question...would you consider introduce feature to lock-down the specified collection data/&quot;query result&quot; in RAM?. F.e. &quot;login_name&quot; or &quot;session&quot; collection would be nice to have always in RAM despite there will be heavy write/read request causing to empty/exchange the whole WT-Cache ?&lt;br/&gt;
To achieve this above we now running 2 mongod in one server, where 1st holds the sensitive-always-to-be-in-RAM data and 2nd mongod hold rest of the data.&lt;/p&gt;

&lt;p&gt;Anyway thanks for update and this can be closed.&lt;/p&gt;

&lt;p&gt;Regards \ Marian&lt;/p&gt;

</comment>
                            <comment id="1396378" author="alexander.gorrod" created="Thu, 29 Sep 2016 03:40:54 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=marian.badinka%40dhl.com&quot; class=&quot;user-hover&quot; rel=&quot;marian.badinka@dhl.com&quot;&gt;marian.badinka@dhl.com&lt;/a&gt; thanks for the suggestion, it is something we have considered in the past. The functionality is not high on our priority list because we have observed that the operating system disk cache can provide most of the benefits a solution implemented inside MongoDB would.&lt;/p&gt;

&lt;p&gt;In cases where a data set fits in RAM when compressed, but does not when uncompressed often the best performance can be achieved by:&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;Ensuring the operating system disk buffer is enabled. On Linux the disk buffer is called the buffer cache: &lt;a href=&quot;http://www.tldp.org/LDP/sag/html/buffer-cache.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://www.tldp.org/LDP/sag/html/buffer-cache.html&lt;/a&gt;&lt;/li&gt;
	&lt;li&gt;Configuring the WiredTiger cache size to be a relatively modest size (in your example I would suggest 1GB). Ideally the WiredTiger cache is still large enough to hold the internal structure and indexes.&lt;/li&gt;
	&lt;li&gt;Monitor I/O statistics to ensure that read operations are not requiring reads from disk.&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;What happens is that WiredTiger will issue operating system read operations for most read operations, but those operations are fast, since they are serviced from memory by the operating system.&lt;/p&gt;</comment>
                            <comment id="1392037" author="thomas.schubert" created="Fri, 23 Sep 2016 15:56:52 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=marian.badinka%40dhl.com&quot; class=&quot;user-hover&quot; rel=&quot;marian.badinka@dhl.com&quot;&gt;marian.badinka@dhl.com&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;Thank you for the improvement request. We have marked this ticket to be considered by the WiredTiger team and we will update the ticket after they have had an opportunity to discuss it.&lt;/p&gt;

&lt;p&gt;Kind regards,&lt;br/&gt;
Thomas&lt;/p&gt;</comment>
                            <comment id="1391808" author="marian.badinka@dhl.com" created="Fri, 23 Sep 2016 12:53:18 +0000"  >&lt;p&gt;not CS ticket please, just normal one&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 23 Sep 2016 15:56:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        7 years, 19 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>alexander.golin@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            7 years, 19 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alexander.gorrod@mongodb.com</customfieldvalue>
            <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>kelsey.schubert@mongodb.com</customfieldvalue>
            <customfieldvalue>marian.badinka@dhl.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjv9z:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hsq2un:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hseikn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>