<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 08:14:16 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[DOCS-15960] [Server] Investigate changes in SERVER-70155: Add duration of how long an oplog slot is kept open to mongod &quot;Slow query&quot; log lines</title>
                <link>https://jira.mongodb.org/browse/DOCS-15960</link>
                <project id="10380" key="DOCS">Documentation</project>
                    <description>    &lt;div class=&quot;panel&quot; style=&quot;background-color: #c2d2c2;border-width: 1px;&quot;&gt;&lt;div class=&quot;panelHeader&quot; style=&quot;border-bottom-width: 1px;background-color: #239eb0;&quot;&gt;&lt;b&gt;Original Downstream Change Summary&lt;/b&gt;&lt;/div&gt;&lt;/div&gt;
&lt;p&gt;    There is a new field called totalOplogSlotDurationMicros logged in &quot;Slow query&quot; messages and written to the database profiler collection, when the value is non-zero.&lt;/p&gt;

&lt;p&gt;As a bit of background, mongod supports parallel writes, but there is a small serialization critical section where writes are handed commit timestamps with which to commit the storage engine writes. Once write operations have commit timestamps, they can commit in any order: out-of-order writes.&lt;/p&gt;

&lt;p&gt;For example,&lt;/p&gt;

&lt;p&gt;writeA receives Timestamp1,&lt;br/&gt;
writeB receives Timestamp2,&lt;br/&gt;
writeC receives Timestamp3,&lt;/p&gt;

&lt;p&gt;then say writeB commits first at Timestamp2. Replication will not proceed to replicate writeB until after writeA commits. writeA is expected to create an oplog entry with Timestamp1 before replication will copy the oplog to secondaries replica set members. This is the idea of an oplog slot. If writeA were to be delayed somehow in committing &amp;#8211; say CPU thread management, or getting stuck in MDB someplace &amp;#8211; then that would delay replication. The new totalOplogSlotDurationMicros value represents the time between a write operation getting a commit timestamp and actually committing, and can help surface issues where operations are committing slowly and slowing down replication.&lt;/p&gt;
&lt;h2&gt;&lt;a name=&quot;DescriptionofLinkedTicket&quot;&gt;&lt;/a&gt;Description of Linked Ticket&lt;/h2&gt;
&lt;p&gt;    An oplog hole represents a Timestamp in the oplog an active storage transaction would commit at that is behind the Timestamp in the oplog an already&amp;#45;committed storage transaction wrote their oplog entry at. For example, it is possible for a storage transaction at Timestamp 20 to have committed and for a storage transaction at Timestamp 10 to commit later on in wall&amp;#45;clock time. Oplog readers are prevented from reading beyond the oplog hole to ensure they don&apos;t miss any oplog entries which might still commit. Keeping an oplog hole open for any extended period of wall&amp;#45;clock time can lead to stalls in replication.&lt;/p&gt;

&lt;p&gt;Vectored insert is an example of an operation which pre&amp;#45;allocates Timestamps to write user data and the corresponding oplog entries at. In MongoDB 4.4, &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-46161&quot; title=&quot;Reduce frequency of yielding during query execution&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-46161&quot;&gt;&lt;del&gt;SERVER-46161&lt;/del&gt;&lt;/a&gt; caused the default internal batch size for vectored insert to increase from 64 to 500. This has been seen to lead to higher tail latencies for vectored inserts (&lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-65054&quot; title=&quot;Avoid slow insert batches blocking replication&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-65054&quot;&gt;SERVER-65054&lt;/a&gt;).&lt;/p&gt;

&lt;p&gt;Other operations cause oplog holes too. If expensive work is done (e.g. within an OpObserver) after the oplog slot is allocated and prior to the storage transaction committing, then those operations can stall replication too. Introducing some logging to track the time spent would give more insight into these areas and perhaps even be useful to signal on within our performance testing.&lt;/p&gt;
</description>
                <environment></environment>
        <key id="2287103">DOCS-15960</key>
            <summary>[Server] Investigate changes in SERVER-70155: Add duration of how long an oplog slot is kept open to mongod &quot;Slow query&quot; log lines</summary>
                <type id="3" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14718&amp;avatarType=issuetype">Task</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="kanchana.sekhar@mongodb.com">Kanchana Sekhar</assignee>
                                    <reporter username="backlog-server-pm">Backlog - Core Eng Program Management Team</reporter>
                        <labels>
                    </labels>
                <created>Mon, 13 Mar 2023 19:23:36 +0000</created>
                <updated>Thu, 11 Jan 2024 23:01:42 +0000</updated>
                            <resolved>Fri, 28 Jul 2023 14:23:59 +0000</resolved>
                                                    <fixVersion>7.0.0-rc0</fixVersion>
                    <fixVersion>6.0.13</fixVersion>
                    <fixVersion>5.0.24</fixVersion>
                    <fixVersion>4.4.29</fixVersion>
                                    <component>manual</component>
                    <component>Server</component>
                        <due></due>
                            <votes>0</votes>
                                    <watches>1</watches>
                                                                                                                <comments>
                            <comment id="5593518" author="kanchana.sekhar" created="Wed, 26 Jul 2023 18:52:04 +0000"  >&lt;p&gt;PR: &lt;a href=&quot;https://github.com/10gen/docs-mongodb-internal/pull/4153&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/10gen/docs-mongodb-internal/pull/4153&lt;/a&gt;&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10320">
                    <name>Documented</name>
                                            <outwardlinks description="documents">
                                        <issuelink>
            <issuekey id="2149660">SERVER-70155</issuekey>
        </issuelink>
                            </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_14258" key="com.atlassian.jira.plugin.system.customfieldtypes:datepicker">
                        <customfieldname>Baseline start date</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 26 Jul 2023 00:00:00 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Wed, 26 Jul 2023 18:52:04 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        28 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_17052" key="com.atlassian.jira.plugin.system.customfieldtypes:textarea">
                        <customfieldname>Downstream Changes Summary</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>There is a new field called totalOplogSlotDurationMicros logged in &amp;quot;Slow query&amp;quot; messages and written to the database profiler collection, when the value is non-zero.&lt;br/&gt;
&lt;br/&gt;
As a bit of background, mongod supports parallel writes, but there is a small serialization critical section where writes are handed commit timestamps with which to commit the storage engine writes. Once write operations have commit timestamps, they can commit in any order: out-of-order writes.&lt;br/&gt;
&lt;br/&gt;
For example,&lt;br/&gt;
&lt;br/&gt;
writeA receives Timestamp1,&lt;br/&gt;
writeB receives Timestamp2,&lt;br/&gt;
writeC receives Timestamp3,&lt;br/&gt;
&lt;br/&gt;
then say writeB commits first at Timestamp2. Replication will not proceed to replicate writeB until after writeA commits. writeA is expected to create an oplog entry with Timestamp1 before replication will copy the oplog to secondaries replica set members. This is the idea of an oplog slot. If writeA were to be delayed somehow in committing -- say CPU thread management, or getting stuck in MDB someplace -- then that would delay replication. The new totalOplogSlotDurationMicros value represents the time between a write operation getting a commit timestamp and actually committing, and can help surface issues where operations are committing slowly and slowing down replication.</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_14873" key="com.atlassian.jira.plugin.system.customfieldtypes:multiuserpicker">
                        <customfieldname>External Reviewer</customfieldname>
                        <customfieldvalues>
                                    <customfieldvalue><![CDATA[gregory.wlodarek@mongodb.com]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>backlog-server-pm</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            28 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                            <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>backlog-server-pm</customfieldvalue>
            <customfieldvalue>kanchana.sekhar@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1zygf:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|i1ic9s:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="7077">CET: Xanadu (25 - 31 Jul 23)</customfieldvalue>
    <customfieldvalue id="7078">CET: Yesterday (1 - 7 Aug 23)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10555" key="com.atlassian.jira.plugin.system.customfieldtypes:float">
                        <customfieldname>Story Points</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|i1zklr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                </customfields>
    </item>
</channel>
</rss>