<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:50:19 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-38879] TTL Index-Only Removal</title>
                <link>https://jira.mongodb.org/browse/SERVER-38879</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Today, when the TTL condition is reached, we remove the old records from the collection and the index.&lt;/p&gt;

&lt;p&gt;With this feature, a configuration option exists to &quot;remove from index only after expiry&quot;.&lt;/p&gt;

&lt;p&gt;Benefit to customer - keep history of TTL collection there, while also having fast indexed queries for recent records.&lt;/p&gt;</description>
                <environment></environment>
        <key id="667205">SERVER-38879</key>
            <summary>TTL Index-Only Removal</summary>
                <type id="2" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14711&amp;avatarType=issuetype">New Feature</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="asya.kamsky@mongodb.com">Asya Kamsky</assignee>
                                    <reporter username="tommyatclassdojo">Tommy Sullivan</reporter>
                        <labels>
                    </labels>
                <created>Mon, 7 Jan 2019 22:03:47 +0000</created>
                <updated>Tue, 5 Feb 2019 23:33:36 +0000</updated>
                            <resolved>Tue, 5 Feb 2019 23:33:36 +0000</resolved>
                                                                    <component>TTL</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="2139050" author="asya" created="Tue, 5 Feb 2019 23:33:36 +0000"  >&lt;p&gt;Closing this as this use case can be handled via partial index - the server does not have another mechanism to remove only a subset of data from an index and TTL feature is intended to affect the documents in the collection, not their index entries.&lt;/p&gt;</comment>
                            <comment id="2110207" author="tommyatclassdojo" created="Wed, 9 Jan 2019 18:07:24 +0000"  >&lt;p&gt;Right. Its cool for us to do something like aging it out by updating the predicate field. I just think it could be a powerful feature for u guys to do that and u may have most of it done already!&lt;/p&gt;

&lt;p&gt;Dynamic expressions in partial queries would be really powerful but may come with some complexities depending on how elaborate the expressions can be! So if that did get released it would probably obviate the need for this feature; but this feature is probably much simpler to release because of its limited complexity (the condition is always just a date aging out, as opposed to more complex expressions)&lt;/p&gt;

&lt;p&gt;I think it would enable your sales guys and gals to make a stronger case for stream-like use cases especially &quot;durable streams&quot;&lt;/p&gt;</comment>
                            <comment id="2108956" author="asya" created="Tue, 8 Jan 2019 18:33:29 +0000"  >&lt;p&gt;&amp;gt; instead we have a boolean field &quot;expired&quot; as our partial index predicate&lt;/p&gt;

&lt;p&gt;This is exactly what we recommend currently - have a job that &quot;ages out&quot; documents by updating a field (&quot;active&quot;) and then partial index can have the predicate &lt;tt&gt;active:true&#160;&lt;/tt&gt; I thought there was already a server ticket to support more &quot;dynamic&quot; type of filter on partial index  I may have been thinking of &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-14784&quot; title=&quot;Add expression indexes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-14784&quot;&gt;SERVER-14784&lt;/a&gt; which is for expression indexes (which is a bit different).&lt;/p&gt;
</comment>
                            <comment id="2108183" author="tommyatclassdojo" created="Tue, 8 Jan 2019 02:49:49 +0000"  >&lt;p&gt;by the way, for context, we are just summing up a bunch of numbers, but doing it only for the recent couple weeks of data, so we want those to be really quickly computable, and so we want them indexed, but since we have years of data in that collection, we don&apos;t need everything in the index, slowing it down, so we really want an index with a moving window of stuff in it. TTL is like a collection with a moving window of stuff in it that is controlled by an index; so we just want a TTL that leaves the collection and only has a moving window on the index...&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;come to think of it you all may be indexing based on everything in the collection and so that may ultimately be why u delete from the collection (so that the index needn&apos;t keep the records)... depending how yall did it the feature may be easier or tougher to implement based on existing TTL implementation&lt;/p&gt;</comment>
                            <comment id="2108182" author="tommyatclassdojo" created="Tue, 8 Jan 2019 02:46:30 +0000"  >&lt;p&gt;from a business perspective u could sell this feature as a way to model durable streams for companies that are already on mongo and thinking of switching to a durable stream technology; instead they could speedily read from recent records but also get to older records without any copying or reindexing or jobs or all that and i don&apos;t know how much it would involve in terms of technical changes on mongo side. hope that helps. thanks again appreciate it.&lt;/p&gt;</comment>
                            <comment id="2108181" author="tommyatclassdojo" created="Tue, 8 Jan 2019 02:43:15 +0000"  >&lt;p&gt;Asya,&lt;/p&gt;

&lt;p&gt;Thank you kindly for your reply. Partial Index, from what I understand, requires that we define a predicate condition. For us, the logical predicate would be something like &quot;document createdAt date field is within the last 2 weeks&quot;.&lt;/p&gt;

&lt;p&gt;The thing is, that the meaning of &quot;within the last two weeks&quot; changes over time. As I understand it, we cannot put a dynamic date expression into our partial index predicate.&lt;/p&gt;

&lt;p&gt;Of course, we have means by which to work around this.&lt;/p&gt;

&lt;p&gt;We could create a partial index today, let that work for a while, and then periodically rebuild the index with an updated date. But that would be taxing on our mongo cluster and impact users. It would also be a little clunky. We may indeed do that if need be but trying to avoid it.&lt;/p&gt;

&lt;p&gt;We could also create a partial index whose predicate is &lt;b&gt;not&lt;/b&gt; dependent on anything dynamic, such as &quot;today&apos;s date&quot;. Perhaps instead we have a boolean field &quot;expired&quot; as our partial index predicate. Then, code in our application could regularly calculate the age of the document and update the field at the appropriate time, causing mongo to take it out of the partial index. Unfortunately, in this solution we have to do a bunch of computation on a schedule just to really update a date counter in order to TTL something. As you can see, this invites us to consider whether the existing TTL feature could be modified to achieve this objective, since it seems to do very close to what we need already.&lt;/p&gt;

&lt;p&gt;The TTL solution actually has exactly what we want, if only it would just do &lt;b&gt;slightly less&lt;/b&gt; than what it already does. If it did everything it already does, but then when it decides an item has expired, it removes it &lt;b&gt;from the index&lt;/b&gt; and not from the collection. That perfectly solves the use case.&lt;/p&gt;

&lt;p&gt;Let me know if that is a reasonable and correct line of thought &lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/smile.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt;&lt;/p&gt;

&lt;p&gt;thanks&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;</comment>
                            <comment id="2108169" author="asya" created="Tue, 8 Jan 2019 01:57:17 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=tommyatclassdojo&quot; class=&quot;user-hover&quot; rel=&quot;tommyatclassdojo&quot;&gt;tommyatclassdojo&lt;/a&gt; what you are describing sounds like the &lt;a href=&quot;https://docs.mongodb.com/manual/core/index-partial/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;partial index feature&lt;/a&gt; - you&apos;re talking about indexing only a subset of (unexpired) documents?&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 8 Jan 2019 01:57:17 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        5 years, 1 week, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            5 years, 1 week, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>asya.kamsky@mongodb.com</customfieldvalue>
            <customfieldvalue>tommyatclassdojo</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|huigqv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hu8h0n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hui307:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>