<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:12:48 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-26665] Clean up WTRecordStore handling of non-oplog capped collections</title>
                <link>https://jira.mongodb.org/browse/SERVER-26665</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;That code dates from a time when we thought that concurrent writes to non-oplog capped collections made sense. Now that we&apos;ve realized that it doesn&apos;t we can significantly simplify the code. All handling of out-of-order commits should only be for the oplog case and not for normal capped collections.&lt;/p&gt;</description>
                <environment></environment>
        <key id="324259">SERVER-26665</key>
            <summary>Clean up WTRecordStore handling of non-oplog capped collections</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="10033" iconUrl="https://jira.mongodb.org/images/icons/statuses/information.png" description="Status for tickets that need to be escalated and unblocked on our team.">Blocked</status>
                    <statusCategory id="4" key="indeterminate" colorName="inprogress"/>
                                    <resolution id="-1">Unresolved</resolution>
                                        <assignee username="backlog-server-execution">Backlog - Storage Execution Team</assignee>
                                    <reporter username="mathias@mongodb.com">Mathias Stearn</reporter>
                        <labels>
                    </labels>
                <created>Mon, 17 Oct 2016 18:46:54 +0000</created>
                <updated>Mon, 4 Dec 2023 19:33:44 +0000</updated>
                                                                            <component>Internal Code</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="4333710" author="daniel.gottlieb@10gen.com" created="Thu, 3 Feb 2022 16:58:27 +0000"  >&lt;p&gt;My point was I don&apos;t know how an application can reliably resume a tailable cursor even on the same* node. Unless the application is willing to table scan from the start to get to its last seen document each time. Happy to be wrong here, I&apos;m not fully aware of how query interacts with non-oplog capped collections.&lt;/p&gt;

&lt;p&gt;Capped collections do serve a purpose when a deployment wants to bound disk usage for a collection such as logs or metrics. I&apos;m not familiar with us developing an analogous behavior elsewhere.&lt;/p&gt;</comment>
                            <comment id="4333520" author="redbeard0531" created="Thu, 3 Feb 2022 16:17:44 +0000"  >&lt;blockquote&gt;
&lt;p&gt;As far as I can tell, a difference in record id ordering would mean that an application cannot resume a tailable cursor across nodes and expect to see all records. But I don&apos;t believe capped collections cursors are reliably resumable by an application today?&lt;/p&gt;&lt;/blockquote&gt;

&lt;p&gt;I think that is basically the only useful feature left in capped collections at this point that you can&apos;t get from non-capped collections. If you don&apos;t need the ability to resume reading a stream, then you probably shouldn&apos;t be using capped collections.&lt;/p&gt;</comment>
                            <comment id="4333447" author="daniel.gottlieb@10gen.com" created="Thu, 3 Feb 2022 15:59:54 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=max.hirschhorn&quot; class=&quot;user-hover&quot; rel=&quot;max.hirschhorn&quot;&gt;max.hirschhorn&lt;/a&gt; showed me we do &lt;a href=&quot;https://github.com/mongodb/mongo/blob/c81c668189f7c7a933661d1e225aacfdbc33840d/src/mongo/db/repl/oplog_applier_utils.cpp#L88-L106&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;serialize writes during oplog application&lt;/a&gt;. So primaries and secondaries should agree on order today &amp;#8211; existing differences were just due to the perceived size of the capped collection. Apologies for the error.&lt;/p&gt;

&lt;p&gt;Clustered indexes not-withstanding, I don&apos;t think we can have nodes agree on oplog order and RecordId order for their capped collection inserts (unless we intend to replicate RecordIds &amp;#8211; which I&apos;ve heard discussed in other contexts).&lt;/p&gt;

&lt;p&gt;While I&apos;m unsure the benefit of unserializing writes to capped collections outweighs the complexities, I&apos;m also not convinced that primaries and secondaries disagreeing on record id order is a deal breaker. Not to dismiss the change entirely, it would certainly be noteworthy. &lt;/p&gt;

&lt;p&gt;As far as I can tell, a difference in record id ordering would mean that an application cannot resume a tailable cursor across nodes and expect to see all records. But I don&apos;t believe capped collections cursors are reliably resumable by an application today?&lt;/p&gt;</comment>
                            <comment id="4333257" author="daniel.gottlieb@10gen.com" created="Thu, 3 Feb 2022 15:19:02 +0000"  >&lt;blockquote&gt;
&lt;p&gt;That solves the biggest issue with unserialized writes then. We would probably also want to ensure that the order on primaries and secondaries is the same since unlike for normal collections, order is considered significant in capped collections.&lt;/p&gt;&lt;/blockquote&gt;
&lt;p&gt;I don&apos;t think serializing writes on the primary today gives us ordering on secondaries, as secondaries can reorder inserts to the same collection. Evidenced (historically as noted) by primaries and secondaries choosing different orders for deletion. I don&apos;t believe allowing concurrent inserts on a primary would change the existing behavior in any meaningful way? Modulo a solution for tailable cursors.&lt;/p&gt;</comment>
                            <comment id="4333076" author="redbeard0531" created="Thu, 3 Feb 2022 14:30:29 +0000"  >&lt;p&gt;That solves the biggest issue with unserialized writes then. We would probably also want to ensure that the order on primaries and secondaries is the same since unlike for normal collections, order is considered significant in capped collections.&lt;/p&gt;</comment>
                            <comment id="4333068" author="milkie" created="Thu, 3 Feb 2022 14:27:58 +0000"  >&lt;p&gt;We explicitly replicate capped deletes now, so primary and secondaries stay in lock step with their capped deletions.&lt;/p&gt;</comment>
                            <comment id="4333062" author="milkie" created="Thu, 3 Feb 2022 14:27:01 +0000"  >&lt;p&gt;I see, you would have to change the behavior of tailable cursors to be read-concern-majority then.  I think that would be palatable to users, and we&apos;d get a performance boost to capped collection writes as well.&lt;/p&gt;</comment>
                            <comment id="4333059" author="redbeard0531" created="Thu, 3 Feb 2022 14:26:48 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt; I think that is what this ticket was initially about. We have all of the logic to support tailable cursors on capped collections with unserialized inserts (or at least what was necessary at the time), however all of that code became unnecessary when we decided to serialize inserts.&lt;/p&gt;

&lt;p&gt;The real issue, and the reason that we decided to serialize was that without serialization, the primary and secondary would choose different documents to be the &quot;oldest&quot; when deleting old documents. I don&apos;t know if we&apos;ve done anything to solve that since then, so maybe that is no longer an issue.&lt;/p&gt;</comment>
                            <comment id="4333046" author="louis.williams" created="Thu, 3 Feb 2022 14:23:38 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=milkie&quot; class=&quot;user-hover&quot; rel=&quot;milkie&quot;&gt;milkie&lt;/a&gt;, yes, if we do &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-18934&quot; title=&quot;Don&amp;#39;t require storage engines to implement oplog visibility rules&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-18934&quot;&gt;SERVER-18934&lt;/a&gt; to implement generic capped collection visibility outside the storage engine, then we get this for free.&lt;/p&gt;</comment>
                            <comment id="4333004" author="milkie" created="Thu, 3 Feb 2022 14:07:23 +0000"  >&lt;p&gt;If we stop serializing capped collection writes, that would make tailable cursor implementation more challenging; do we have a plan for that or would we stop supporting them?&lt;/p&gt;</comment>
                            <comment id="4327956" author="louis.williams" created="Tue, 1 Feb 2022 16:45:58 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=redbeard0531&quot; class=&quot;user-hover&quot; rel=&quot;redbeard0531&quot;&gt;redbeard0531&lt;/a&gt;, does the motivation in this ticket still hold? We think we may want to reconsider serializing capped collection writes in the future.&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10011">
                    <name>Depends</name>
                                            <outwardlinks description="depends on">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_12751" key="com.atlassian.jira.plugin.system.customfieldtypes:multiselect">
                        <customfieldname>Assigned Teams</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="25136"><![CDATA[Storage Execution]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Thu, 27 Jan 2022 19:46:52 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        2 years, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[<a href='https://jira.mongodb.org/browse/PM-2983'>PM-2983</a>]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>steven.vannelli@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            2 years, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-execution</customfieldvalue>
            <customfieldvalue>daniel.gottlieb@mongodb.com</customfieldvalue>
            <customfieldvalue>milkie@mongodb.com</customfieldvalue>
            <customfieldvalue>louis.williams@mongodb.com</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrjt6v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hyw432:w</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsefu7:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>