<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:18:01 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-8663] Slow performance on duplicate key exception</title>
                <link>https://jira.mongodb.org/browse/SERVER-8663</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;While doing testing with C++ driver, I noticed that when all&lt;br/&gt;
keys were duplicates the performance was way slower.&lt;br/&gt;
Doesnt make sense since it&apos;s not supposed to do work in that case...&lt;/p&gt;

&lt;p&gt;I tried with the very simple loop in js, and gives even worse result:&lt;/p&gt;

&lt;p&gt;var start = new Date();&lt;br/&gt;
for (var count = 0; count &amp;lt; 1000000; ++count) {&lt;br/&gt;
db.foo.insert({_id: count});&lt;br/&gt;
}&lt;br/&gt;
print(new Date() - start);&lt;/p&gt;

&lt;p&gt;Collection is empty:&lt;br/&gt;
antoine@ag410:~/Downloads/mongodb-linux-x86_64-2.2.2$ ./bin/mongo&lt;br/&gt;
~/adobe/testwrites.js&lt;br/&gt;
MongoDB shell version: 2.2.2&lt;br/&gt;
connecting to: test&lt;br/&gt;
12527&lt;/p&gt;

&lt;p&gt;2nd run without dropping:&lt;br/&gt;
antoine@ag410:~/Downloads/mongodb-linux-x86_64-2.2.2$ ./bin/mongo&lt;br/&gt;
~/adobe/testwrites.js&lt;br/&gt;
MongoDB shell version: 2.2.2&lt;br/&gt;
connecting to: test&lt;br/&gt;
78153&lt;/p&gt;

&lt;p&gt;this is reproducible always..&lt;/p&gt;</description>
                <environment></environment>
        <key id="66061">SERVER-8663</key>
            <summary>Slow performance on duplicate key exception</summary>
                <type id="4" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14710&amp;avatarType=issuetype">Improvement</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="2">Won&apos;t Fix</resolution>
                                        <assignee username="mathias@mongodb.com">Mathias Stearn</assignee>
                                    <reporter username="antoine">Antoine Girbal</reporter>
                        <labels>
                    </labels>
                <created>Fri, 22 Feb 2013 00:53:46 +0000</created>
                <updated>Thu, 14 May 2015 17:59:27 +0000</updated>
                            <resolved>Thu, 14 May 2015 17:59:27 +0000</resolved>
                                    <version>2.2.2</version>
                                                    <component>Performance</component>
                                        <votes>0</votes>
                                    <watches>3</watches>
                                                                                                                <comments>
                            <comment id="273429" author="redbeard0531" created="Fri, 22 Feb 2013 21:01:09 +0000"  >&lt;p&gt;&lt;a href=&quot;https://jira.mongodb.org/secure/ViewProfile.jspa?name=antoine&quot; class=&quot;user-hover&quot; rel=&quot;antoine&quot;&gt;antoine&lt;/a&gt;, That case only comes up in the rare case that we can&apos;t do two-phase indexing. In the normal case we never need to roll-back the insert since it hasn&apos;t occured yet (we check if the insert will work before doing it).&lt;/p&gt;

&lt;p&gt;My profiling suggests that all of the overhead is due to exception-unwind overhead. Most C++ compliers use &quot;zero-cost&quot; exceptions which makes it free to enter try-block and move all of the work of figuring out how to handle the exception into the throw case. This extra work is why the dup-key case is slower than then clean insert case. We may be able to optimize some of this away, but I don&apos;t know if it is worth it as I&apos;m able to do ~20k dup-key inserts per second on my machine. Anything even approaching that number is likely to be a serious bug in user code anyway.&lt;/p&gt;</comment>
                            <comment id="272659" author="antoine" created="Fri, 22 Feb 2013 00:56:07 +0000"  >&lt;p&gt;I&apos;ve tried to profile with gprof but mongod keeps hanging after a few&lt;br/&gt;
thousand records when testing duplicate keys.&lt;br/&gt;
Attached are the stack traces of the hung process as well as the pdf&lt;br/&gt;
result for a few records worth.&lt;/p&gt;

&lt;p&gt;Looking at our code a bit, I see on dup key we do this:&lt;/p&gt;

&lt;p&gt;                   // normal case &amp;#8211; we can roll back&lt;br/&gt;
                    _deleteRecord(d, ns, r, loc);&lt;br/&gt;
                    throw;&lt;/p&gt;

&lt;p&gt;Is it expected to be slower, since we actually insert then remove the&lt;br/&gt;
record on dup key?&lt;br/&gt;
I guess we need to insert first in order to get a document location&lt;br/&gt;
for the index to use..&lt;/p&gt;</comment>
                            <comment id="272658" author="antoine" created="Fri, 22 Feb 2013 00:55:03 +0000"  >&lt;p&gt;Reproduced with 2.0 and 2.2.&lt;br/&gt;
I tried doing upsert instead, it&apos;s a bit faster, but still x2 slower&lt;br/&gt;
than straight inserts&lt;br/&gt;
Rates / s below&lt;/p&gt;

&lt;p&gt;straight inserts (no preexisting):&lt;br/&gt;
20095: aggregate rate = 16902 17273 17236 16697 16437 16788&lt;/p&gt;

&lt;p&gt;inserts all dups:&lt;br/&gt;
20195: aggregate rate = 5804 7951 5793 5506 5536 4993&lt;/p&gt;

&lt;p&gt;upserts:&lt;br/&gt;
20061: aggregate rate = 9541 10811 8811 8852 6541 8242 8125&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                            <attachment id="24312" name="dups.pdf" size="17403" author="antoine" created="Fri, 22 Feb 2013 00:56:58 +0000"/>
                            <attachment id="24311" name="stacks.txt" size="24802" author="antoine" created="Fri, 22 Feb 2013 00:56:58 +0000"/>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>3.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Fri, 22 Feb 2013 21:01:09 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 51 weeks, 5 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>false</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 51 weeks, 5 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>antoine</customfieldvalue>
            <customfieldvalue>mathias@mongodb.com</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrn5bb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrg1t3:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7044</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs9zfj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>