<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 04:42:47 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-36330] Total transaction operations size only 16MB?</title>
                <link>https://jira.mongodb.org/browse/SERVER-36330</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;In my application I&apos;m trying to make use of newly added multi-document transactions feature. &lt;/p&gt;

&lt;p&gt;I got the following error:&lt;/p&gt;

&lt;p&gt;&lt;font color=&quot;red&quot;&gt;&lt;tt&gt;Total size of all transaction operations must be less than 16793600. Actual size is 16793817&lt;/tt&gt;&lt;/font&gt;&lt;/p&gt;

&lt;p&gt;Why this limit was not specified in documentation? And do you have plans to increase the limit?&lt;/p&gt;

&lt;p&gt;During testing I also run into several other problems, like unexpected &quot;Transaction aborted&quot; errors when I had a few thousands of simultaneously open server sessions. Is there any document (like blogpost or maybe some internal docs) with a description of all limitations?&lt;/p&gt;</description>
                <environment>NodeJS driver v3.1.0&lt;br/&gt;
Windows 10 x64</environment>
        <key id="577697">SERVER-36330</key>
            <summary>Total transaction operations size only 16MB?</summary>
                <type id="6" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14720&amp;avatarType=issuetype">Question</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="13202">Works as Designed</resolution>
                                        <assignee username="alyson.cabral@mongodb.com">Alyson Cabral</assignee>
                                    <reporter username="kreig">Vyacheslav Stroy</reporter>
                        <labels>
                    </labels>
                <created>Fri, 27 Jul 2018 20:12:29 +0000</created>
                <updated>Fri, 27 Oct 2023 13:53:38 +0000</updated>
                            <resolved>Mon, 30 Jul 2018 15:02:28 +0000</resolved>
                                    <version>4.0.0</version>
                                                    <component>Replication</component>
                                        <votes>0</votes>
                                    <watches>11</watches>
                                                                                                                <comments>
                            <comment id="2341544" author="grigori.melnik" created="Wed, 24 Jul 2019 14:43:56 +0000"  >&lt;p&gt;Check out v4.2 which supports large transactions over multiple oplog entries.&lt;/p&gt;</comment>
                            <comment id="2275388" author="rupesm1993" created="Fri, 7 Jun 2019 06:25:39 +0000"  >&lt;p&gt;&#160;Hi Aly,&#160;&lt;/p&gt;

&lt;p&gt;Is it not that we can change the oplog size as mentioned in the below link&#160;&lt;/p&gt;

&lt;p&gt;&lt;a href=&quot;https://docs.mongodb.com/manual/tutorial/change-oplog-size/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;https://docs.mongodb.com/manual/tutorial/change-oplog-size/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;If yes, then does it means we can overcome this transaction limit size by changing the oplog size?&lt;/p&gt;

&lt;p&gt;If no, then are oplogs mentioned in the above link is different than the oplog entry which we are discussing in this thread so far? If so, what is the exact difference between both these oplogs?&lt;/p&gt;</comment>
                            <comment id="2248630" author="kelvinmuia@gmail.com" created="Thu, 16 May 2019 11:31:51 +0000"  >&lt;p&gt;Any update on removing the transaction size constraint&lt;/p&gt;</comment>
                            <comment id="2221083" author="alyson.cabral" created="Mon, 22 Apr 2019 20:21:57 +0000"  >&lt;p&gt;Hi &lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=rupesm1993&quot; class=&quot;user-hover&quot; rel=&quot;rupesm1993&quot;&gt;rupesm1993&lt;/a&gt;,&lt;/p&gt;

&lt;p&gt;In 4.0, each transaction has an oplog entry that consists of all the deltas changed to the document (or row), not the full document (or row). For example, if you&apos;re doing many small updates within a single transaction, only the fields changed will be represented in the 16MB limit oplog entry.  However if you&apos;re doing many inserts within a single transaction, each of the full documents will be stored in that 16MB entry. &lt;/p&gt;

&lt;p&gt;We are actively working on removing this size constraint. &lt;/p&gt;</comment>
                            <comment id="2216292" author="rupesm1993" created="Wed, 17 Apr 2019 12:43:11 +0000"  >&lt;p&gt;Hi, We have 1 million&#160; rows in MySQL and we want to migrate to MongoDB, but this limit of 16 MB inside the transaction is making MongoDB impractical I guess. Because if any one want to use transaction, probability of data having greater than 16 MB is always greater.&lt;/p&gt;

&lt;p&gt;&#160;&lt;/p&gt;

&lt;p&gt;Let me know if I am missing something as per the docs it&apos;s the document size which is 16 MB and document means rows in MySQL and my rows size is no where greater than 16 MB.&#160;&lt;/p&gt;

&lt;p&gt;If this size is sum of the all rows in transaction, can you know if anyone has been able to use this transaction feature with any data greater than some million rows?&lt;/p&gt;

&lt;p&gt;Even if it will change to 64 MB, will it be of any help. Kindly help&lt;/p&gt;</comment>
                            <comment id="2016107" author="alyson.cabral" created="Thu, 27 Sep 2018 13:03:12 +0000"  >&lt;p&gt;Not just yet. Please join the beta group linked in my previous comment and I&apos;ll notify you as soon as it&apos;s available to test.&lt;/p&gt;</comment>
                            <comment id="2016029" author="daniel.moqvist@knowit.se" created="Thu, 27 Sep 2018 11:10:06 +0000"  >&lt;p&gt;Hi, today we encountered the 16Mb limitaton in our transactions.&#160;&lt;/p&gt;

&lt;p&gt;Do you have any updated status about the fix for this transaction limitation?&lt;/p&gt;

&lt;p&gt;Is there any beta or rc we can test?&lt;/p&gt;</comment>
                            <comment id="1960182" author="alyson.cabral" created="Mon, 30 Jul 2018 15:02:12 +0000"  >&lt;p&gt;I&apos;m going to close this ticket as works as designed but feel free to open a separate server ticket if you do find unexpected behavior. &lt;/p&gt;

&lt;p&gt;Also, I recommend you join the &lt;a href=&quot;https://groups.google.com/forum/#!forum/mongodb-txnbeta&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;transactions beta group&lt;/a&gt; and ask any development questions related to transactions. &lt;/p&gt;</comment>
                            <comment id="1959130" author="kreig" created="Fri, 27 Jul 2018 21:23:12 +0000"  >&lt;p&gt;Hey @Aly,&lt;/p&gt;

&lt;p&gt;Thanks for the reference and clarification of implementation details.&lt;/p&gt;

&lt;p&gt;Will get back to you with detailed stack traces once I run into the mentioned error again.&lt;/p&gt;</comment>
                            <comment id="1959100" author="alyson.cabral" created="Fri, 27 Jul 2018 20:56:57 +0000"  >&lt;p&gt;Hi there,&lt;/p&gt;

&lt;p&gt;This limit is outlined in the documentation under &lt;a href=&quot;https://docs.mongodb.com/manual/core/transactions-production-consideration/&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;oplog size limit&lt;/a&gt; on the production considerations page.&lt;/p&gt;

&lt;p&gt;In MongoDB 4.0 the entire description of modifications in a transaction (i.e. changes made by each statement) are represented as a single oplog entry. Not even oplog entries are immune to 16MB document size limit upheld by the system. While this limitation does encourage developers to conform to MongoDB best practices in some cases, it is a hard limitation to reason about, as the combined size of the description of changes should be an implementation detail for many users. It is a priority for us to remove this limitation by allowing transactions to be represented across multiple oplog entries.&lt;/p&gt;

&lt;p&gt;I&apos;ll update the &lt;a href=&quot;https://groups.google.com/forum/#!forum/mongodb-txnbeta&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;transactions beta group&lt;/a&gt; as soon as we have the functionality available to test. Also, feel free to ask any transactions usability questions there. I&apos;d love to hear more about the unexpected aborted transactions you are encountering. Keep in mind that transactions are aborted when they encounter write conflicts, so if you are doing concurrent writes to the same set of documents that could be a reason.&lt;/p&gt;

&lt;p&gt;As an additional resource, I gave a &lt;a href=&quot;https://www.youtube.com/watch?v=3x_Pf9rQGCo&amp;amp;t=134s&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;talk at MongoDB World&lt;/a&gt; that goes over limitations and guarantees.&lt;/p&gt;

&lt;p&gt;Aly Cabral&lt;br/&gt;
MongoDB Product Manager, Distributed Systems&lt;/p&gt;</comment>
                    </comments>
                <issuelinks>
                            <issuelinktype id="10420">
                    <name>Backports</name>
                                                                <inwardlinks description="backports">
                                        <issuelink>
            <issuekey id="581699">SERVER-36428</issuekey>
        </issuelink>
                            </inwardlinks>
                                    </issuelinktype>
                    </issuelinks>
                <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>10.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 27 Jul 2018 20:40:02 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        4 years, 29 weeks ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>luke.bonanomi@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            4 years, 29 weeks ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                    <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>alyson.cabral@mongodb.com</customfieldvalue>
            <customfieldvalue>daniel.moqvist@knowit.se</customfieldvalue>
            <customfieldvalue>grigori.melnik@mongodb.com</customfieldvalue>
            <customfieldvalue>kelvinmuia@gmail.com</customfieldvalue>
            <customfieldvalue>rupesm1993</customfieldvalue>
            <customfieldvalue>kreig</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu3onz:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|htucpj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>9223372036854775807</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hu3axb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>