<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 06:20:22 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-71934] Only reap idents once the pinned timestamp passes the drop timestamp</title>
                <link>https://jira.mongodb.org/browse/SERVER-71934</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;WiredTiger does not support transactional DDL operations - so when an ident in WiredTiger is dropped, the data is deleted immediately, and it is not possible to read at a timestamp prior to the drop timestamp. &lt;/p&gt;

&lt;p&gt;In order to allow MongoDB users to read at a timestamp prior to the drop timestamp, we implement &quot;two-phase dropping&quot;, whereby a collection is removed from the catalog, and the actual data for the collection (including the ident for the collection and idents for all its indexes) is deleted later. This works as follows:&lt;/p&gt;

&lt;p&gt;When a user drops a collection, the ident for the collection (and the idents for its indexes) are added to a &quot;drop pending ident reaper&quot;, which holds a list of idents to be dropped. When the minimum of the oldest timestamp and the checkpoint timestamp in WiredTiger advances, we &lt;a href=&quot;https://github.com/10gen/mongo/blob/1c32419a4c630570250837bba4a845ffdca30a78/src/mongo/db/storage/storage_engine_impl.cpp#L1266&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;call DropPendingIdentReaper::dropIdentsOlderThan&lt;/a&gt; to drop idents older than that timestamp.&lt;/p&gt;

&lt;p&gt;However, to keep an ident from being dropped while reads are in progress on that ident, the drop pending ident reaper also &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1c32419a4c630570250837bba4a845ffdca30a78/src/mongo/db/storage/kv/kv_drop_pending_ident_reaper.h#L131-L133&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;keeps a weak_ptr to an Ident object&lt;/a&gt;, and &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1c32419a4c630570250837bba4a845ffdca30a78/src/mongo/db/storage/kv/kv_drop_pending_ident_reaper.cpp#L145&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;avoids actually dropping the ident&lt;/a&gt; while outstanding references to this ident remain. These references exist for collection idents in the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/402b2fac002941eeb67da425fc16264259226267/src/mongo/db/storage/record_store.h#L734&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;RecordStore&lt;/a&gt; owned by instances of the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/1c32419a4c630570250837bba4a845ffdca30a78/src/mongo/db/catalog/collection_impl.h#L400&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CollectionImpl class&lt;/a&gt;. For indexes, these references exist in the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69780bf05606a36c56cf90464970ca72182be6b/src/mongo/db/storage/sorted_data_interface.h#L409&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SortedDataInterface&lt;/a&gt;, which is owned by the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69780bf05606a36c56cf90464970ca72182be6b/src/mongo/db/index/index_access_method.h#L666&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;IndexAccessMethod&lt;/a&gt;, which is owned by the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69780bf05606a36c56cf90464970ca72182be6b/src/mongo/db/catalog/index_catalog_entry_impl.h#L224&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;IndexCatalogEntry&lt;/a&gt;, which is owned by &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69780bf05606a36c56cf90464970ca72182be6b/src/mongo/db/catalog/index_catalog_impl.h#L418-L420&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;IndexCatalogImpl&lt;/a&gt;, which is owned by the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/a69780bf05606a36c56cf90464970ca72182be6b/src/mongo/db/catalog/collection_impl.h#L442&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;CollectionImpl&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Managing these references is a bit tricky, and WiredTiger already has a timestamp called the &apos;pinned timestamp&apos;, which is the &lt;a href=&quot;https://source.wiredtiger.com/develop/timestamp_global_api.html&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;&quot;minimum of the oldest_timestamp and the oldest active reader&quot;&lt;/a&gt;. If, instead of dropping idents when the oldest_timestamp advanced, we waited until the pinned_timestamp advanced, we could remove the need for reference counting ident usage inside MongoDB and simplify this whole garbage collection process.&lt;/p&gt;

&lt;p&gt;One possible downside is that the pinned timestamp is controlled by readers across all collections, so if there is a reader for a timestamp older than the oldest_timestamp for one collection, this will prevent dropping the ident for another collection which was dropped before the oldest_timestamp and has no remaining readers.&lt;/p&gt;</description>
                <environment></environment>
        <key id="2205332">SERVER-71934</key>
            <summary>Only reap idents once the pinned timestamp passes the drop timestamp</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="12300">Won&apos;t Do</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="matthew.saltz@mongodb.com">Matthew Saltz</reporter>
                        <labels>
                    </labels>
                <created>Wed, 7 Dec 2022 19:24:27 +0000</created>
                <updated>Tue, 13 Dec 2022 16:44:38 +0000</updated>
                            <resolved>Tue, 13 Dec 2022 16:44:37 +0000</resolved>
                                                                                        <votes>0</votes>
                                    <watches>2</watches>
                                                                                                                <comments>
                            <comment id="5051646" author="JIRAUSER1258161" created="Tue, 13 Dec 2022 16:44:38 +0000"  >&lt;p&gt;Unfortunately, we can&apos;t do this due to untimestamped reads.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Tue, 13 Dec 2022 16:44:38 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        1 year, 8 weeks, 1 day ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>dan.larkin-york@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            1 year, 8 weeks, 1 day ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>dan.larkin-york@mongodb.com</customfieldvalue>
            <customfieldvalue>matthew.saltz@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1lv13:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i14fog:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1lh6f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>