<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:54:42 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-20602] Malformed chunk at end of metrics file</title>
                <link>https://jira.mongodb.org/browse/SERVER-20602</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Some metrics files end with a malformed chunk having an empty reference document but nmetrics &amp;gt; 0. Ideally an empty chunk shouldn&apos;t be present at all, but if it is the nmetrics value should be 0.&lt;/p&gt;</description>
                <environment></environment>
        <key id="231286">SERVER-20602</key>
            <summary>Malformed chunk at end of metrics file</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="andrew.morrow@mongodb.com">Andrew Morrow</assignee>
                                    <reporter username="bruce.lucas@mongodb.com">Bruce Lucas</reporter>
                        <labels>
                            <label>FTDC</label>
                    </labels>
                <created>Thu, 24 Sep 2015 11:30:52 +0000</created>
                <updated>Wed, 7 Oct 2015 21:35:48 +0000</updated>
                            <resolved>Thu, 1 Oct 2015 15:41:59 +0000</resolved>
                                                    <fixVersion>3.1.9</fixVersion>
                                    <component>Diagnostics</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="1048757" author="xgen-internal-githook" created="Thu, 1 Oct 2015 15:41:15 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;acmorrow&apos;, u&apos;name&apos;: u&apos;Andrew Morrow&apos;, u&apos;email&apos;: u&apos;acm@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-20602&quot; title=&quot;Malformed chunk at end of metrics file&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-20602&quot;&gt;&lt;del&gt;SERVER-20602&lt;/del&gt;&lt;/a&gt; There is never FTDC data to flush if we lack a reference document&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/a861bf352a9e71ac46e80d19d96f5fa5d9c068e3&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/a861bf352a9e71ac46e80d19d96f5fa5d9c068e3&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1046082" author="bruce.lucas@10gen.com" created="Tue, 29 Sep 2015 14:03:11 +0000"  >&lt;p&gt;Took a closer look at the data, and as far as I can see with this fix there are no missing samples.&lt;/p&gt;</comment>
                            <comment id="1045291" author="acm" created="Mon, 28 Sep 2015 19:15:48 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bruce.lucas%40mongodb.com&quot; class=&quot;user-hover&quot; rel=&quot;bruce.lucas@mongodb.com&quot;&gt;bruce.lucas@mongodb.com&lt;/a&gt; - I&apos;ve confirmed that it is the same issue. The behavior here is that you add a sample, which puts you over the sample limit, so you flush some data and reset the reference document. Flushing that data puts you over the file size limit, so you go to rotate. Rotate closes the file, which calls flush again. At this point, due to the bug identified before, we interpret the non-reset value for _compressedChunkBuffer.len() &amp;gt; 0 as indicating that we have data, even though the reference doc is empty. At that point, we write a malformed chunk. I&apos;m pretty confident that this is the right fix. My only concern, which I share with you, is that there is some other condition that necessitates the second part of that expression that we are overlooking. If you could take a closer look at your data and try to establish whether it is complete, I&apos;d be more comfortable proposing this as the fix.&lt;/p&gt;</comment>
                            <comment id="1044814" author="bruce.lucas@10gen.com" created="Mon, 28 Sep 2015 14:06:17 +0000"  >&lt;p&gt;The issue I observed occurs when the file size reaches its limit and we roll over to the next file. I don&apos;t know whether it also occurs on shutdown - it may. I tried the change you identified and it does indeed seem to fix the issue that I observed on file rollover, and a cursor look shows the files are otherwise still valid (although I haven&apos;t looked closely to make sure for example that we aren&apos;t losing samples at the point where we roll over).&lt;/p&gt;</comment>
                            <comment id="1043921" author="acm" created="Fri, 25 Sep 2015 20:46:19 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=bruce.lucas&quot; class=&quot;user-hover&quot; rel=&quot;bruce.lucas&quot;&gt;bruce.lucas&lt;/a&gt; - I believe I have diagnosed this issue. Here is the offending line:&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://github.com/mongodb/mongo/blob/88c34d5fe384f80508c3b489ced5fcf4ae914bd1/src/mongo/db/ftdc/compressor.h#L113&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/blob/88c34d5fe384f80508c3b489ced5fcf4ae914bd1/src/mongo/db/ftdc/compressor.h#L113&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The first part of that expression is fine: if we don&apos;t have a reference document, we definitely don&apos;t have data. However the or-ed in check of the length of the uncompressed data is erroneous. After flushing data, the compressor resets the reference document. However, it does not reset other state variables. At shutdown, mongod does a final flush. If that shutdown flush is invoked after a flush driven by the addition of new data without an intervening sample collection, then the reference document will be empty, but the length of the _compressedChunkBuffer will be non-zero. This causes us to actually attempt to write data, with an empty reference document and a non-zero metrics count. This would also explain why only some metrics files have this issue: mongod needs to be terminated right after it has flushed a file, and before it has collected another sample.&lt;/p&gt;

&lt;p&gt;The fix appears to be simply to remove the &quot; || _compressedChunkBuffer.len() &amp;gt; 0&quot; from that line, and allow the presence or absence of the reference document to be the sole determinant of whether there is data to be flushed. This makes sense: if you have a reference document, you have at least one datum being the reference document itself, if you don&apos;t have a reference document, you don&apos;t have anything to flush.&lt;/p&gt;

&lt;p&gt;Could you please test the above change locally and see if it fixes the issue for you? If so I will get it up for CR and patch queued.&lt;/p&gt;
</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 25 Sep 2015 20:46:19 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        8 years, 19 weeks, 6 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10857" key="com.pyxis.greenhopper.jira:gh-epic-link">
                        <customfieldname>Epic Link</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>PM-176</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            8 years, 19 weeks, 6 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>andrew.morrow@mongodb.com</customfieldvalue>
            <customfieldvalue>bruce.lucas@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrktpz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrenev:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="688">Platform A (10/09/15)</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hsfmmv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>