<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:20:24 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-29282] BSON Document Size can be exceeded when grouping inserts on SECONDARY nodes</title>
                <link>https://jira.mongodb.org/browse/SERVER-29282</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;During steady state replication, due to the way that &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1084-L1098&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;SECONDARY nodes group &apos;insert&apos; oplog entries&lt;/a&gt;, the BSON maximum document size of 16MB may be exceeded if we try to group together a batch that include documents at or near the 16MB limit.&lt;/p&gt;

&lt;p&gt;The &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1092-L1097&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;predicate&lt;/a&gt; for how we delimit a group cuts off the group at the first op that exceeds the group size limit, but when we put these ops into a group &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1113&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;, it could put us over a maximum document size when we try to construct a BSON object &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1118&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;here&lt;/a&gt;. This is due to the incorrect way that we measure the size of a batch. Given a list of ops, we &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1089&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;start iterating from the second op&lt;/a&gt;, and initialize the &lt;a href=&quot;https://github.com/mongodb/mongo/blob/r3.5.7/src/mongo/db/repl/sync_tail.cpp#L1086&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;batchSize to zero&lt;/a&gt;. Thus, we don&apos;t actually include the size of the first op in our measurement of the batch size, which is how we could be allowed to exceed document size limits when our ops are very close to 16MB.&lt;/p&gt;

&lt;p&gt;The logic in place to handle insert grouping is somewhat hard to follow as is. This bug can be addressed by limiting batch sizes correctly by including the sizes of all ops, and making sure we delimit the group correctly based on this.  Ideally, the grouping logic can be cleaned up at the same time to make it more clear that is correct.&lt;/p&gt;

&lt;p&gt;A JS test reproducing this issue is attached. It reproduced the issue reliably running on MongoDB 3.5.7.&lt;/p&gt;</description>
                <environment></environment>
        <key id="384838">SERVER-29282</key>
            <summary>BSON Document Size can be exceeded when grouping inserts on SECONDARY nodes</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13201">Fixed</resolution>
                                        <assignee username="william.schultz@mongodb.com">William Schultz</assignee>
                                    <reporter username="william.schultz@mongodb.com">William Schultz</reporter>
                        <labels>
                    </labels>
                <created>Thu, 18 May 2017 18:25:11 +0000</created>
                <updated>Mon, 30 Oct 2023 23:16:40 +0000</updated>
                            <resolved>Thu, 13 Jul 2017 22:24:38 +0000</resolved>
                                                    <fixVersion>3.4.7</fixVersion>
                    <fixVersion>3.5.11</fixVersion>
                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>8</watches>
                                                                                                                <comments>
                            <comment id="1636693" author="xgen-internal-githook" created="Mon, 31 Jul 2017 19:07:46 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{&apos;email&apos;: &apos;william.schultz@mongodb.com&apos;, &apos;username&apos;: &apos;will62794&apos;, &apos;name&apos;: &apos;William Schultz&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-29282&quot; title=&quot;BSON Document Size can be exceeded when grouping inserts on SECONDARY nodes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-29282&quot;&gt;&lt;del&gt;SERVER-29282&lt;/del&gt;&lt;/a&gt; Prevent BSON Document size from being exceeded when grouping inserts on SECONDARY&lt;/p&gt;

&lt;p&gt;(cherry picked from commit 1da62c11258aaa91dcff3f0133775aae615e29d4)&lt;br/&gt;
Branch: v3.4&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/aadff4eb0b8b051777a6e4315e91dde83369e2cd&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/aadff4eb0b8b051777a6e4315e91dde83369e2cd&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1621686" author="xgen-internal-githook" created="Thu, 13 Jul 2017 22:23:23 +0000"  >&lt;p&gt;Author:&lt;/p&gt;
{u&apos;username&apos;: u&apos;will62794&apos;, u&apos;name&apos;: u&apos;William Schultz&apos;, u&apos;email&apos;: u&apos;william.schultz@mongodb.com&apos;}
&lt;p&gt;Message: &lt;a href=&quot;https://jira.mongodb.org/browse/SERVER-29282&quot; title=&quot;BSON Document Size can be exceeded when grouping inserts on SECONDARY nodes&quot; class=&quot;issue-link&quot; data-issue-key=&quot;SERVER-29282&quot;&gt;&lt;del&gt;SERVER-29282&lt;/del&gt;&lt;/a&gt; Prevent BSON Document size from being exceeded when grouping inserts on SECONDARY&lt;br/&gt;
Branch: master&lt;br/&gt;
&lt;a href=&quot;https://github.com/mongodb/mongo/commit/1da62c11258aaa91dcff3f0133775aae615e29d4&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/1da62c11258aaa91dcff3f0133775aae615e29d4&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1577269" author="spencer" created="Mon, 22 May 2017 22:30:43 +0000"  >&lt;p&gt;If this is a regression in 3.4 then we probably want to try to fix this soon and get it backported.&lt;/p&gt;</comment>
                            <comment id="1575364" author="william.schultz" created="Fri, 19 May 2017 14:19:46 +0000"  >&lt;p&gt;Yes, I was able to reproduce this on 3.4.4. Not 3.2 or 3.0 though. This looks like the commit that introduced the issue: &lt;a href=&quot;https://github.com/mongodb/mongo/commit/2e153f35f45e284d066210792b7b231b033baaa8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://github.com/mongodb/mongo/commit/2e153f35f45e284d066210792b7b231b033baaa8&lt;/a&gt;&lt;/p&gt;</comment>
                            <comment id="1575067" author="schwerin" created="Fri, 19 May 2017 01:08:50 +0000"  >&lt;p&gt;Does this affect any of the stable branches?&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                            <outwardlinks description="backported by">
                                                        </outwardlinks>
                                                        </issuelinktype>
                            <issuelinktype id="10010">
                    <name>Duplicate</name>
                                                                <inwardlinks description="is duplicated by">
                                        <issuelink>
            <issuekey id="477437">SERVER-32499</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                            <issuelinktype id="10012">
                    <name>Related</name>
                                            <outwardlinks description="related to">
                                                        </outwardlinks>
                                                        </issuelinktype>
                    </issuelinks>
                <attachments>
                            <attachment id="156482" name="docsize.js" size="2273" author="william.schultz@mongodb.com" created="Thu, 18 May 2017 18:27:26 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>5.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18555" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname># of Sprints</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2.0</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_12450" key="com.atlassian.jira.plugin.system.customfieldtypes:multicheckboxes">
                        <customfieldname>Backport Requested</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="14340"><![CDATA[v3.4]]></customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10011" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Backwards Compatibility</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10038"><![CDATA[Fully Compatible]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                    <customfield id="customfield_13552" key="com.go2group.jira.plugin.crm:crm_generic_field">
                        <customfieldname>Case</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[[500A000000X10ANIAZ, 500A000000ZdJteIAF, 500A000000a8afZIAQ]]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 19 May 2017 01:08:50 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        6 years, 28 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            6 years, 28 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>schwerin@mongodb.com</customfieldvalue>
            <customfieldvalue>xgen-internal-githook</customfieldvalue>
            <customfieldvalue>spencer@mongodb.com</customfieldvalue>
            <customfieldvalue>william.schultz@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht7nen:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|ht1a8n:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_10557" key="com.pyxis.greenhopper.jira:gh-sprint">
                        <customfieldname>Sprint</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue id="1712">Repl 2017-06-19</customfieldvalue>
    <customfieldvalue id="1768">Repl 2017-07-31</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht494v:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>