<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:00:29 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-2593] Background index creation inflates concurrent inserts/s operations in mongostat</title>
                <link>https://jira.mongodb.org/browse/SERVER-2593</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;(Copied from &lt;a href=&quot;http://groups.google.com/group/mongodb-user/browse_thread/thread/f65c2178ade3f9c8&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://groups.google.com/group/mongodb-user/browse_thread/thread/f65c2178ade3f9c8&lt;/a&gt;)&lt;/p&gt;

&lt;p&gt;Hi,&lt;/p&gt;

&lt;p&gt;I was just playing with MongoDB 1.6.5 (and development 1.7.6) and I&lt;br/&gt;
notice that if I am inserting a lot of items concurrently (using a for&lt;br/&gt;
loop at a mongo console) and perform a background index, the inserts/s&lt;br/&gt;
figure increases while background indexing. I am on Windows 7 64bit.&lt;/p&gt;

&lt;p&gt;Is it really increasing or is there some bug here?&lt;/p&gt;

&lt;p&gt;1) Start off with this running on a separate console:&lt;br/&gt;
&amp;gt; for(var i = 1; i &amp;lt; 10000000; i++) db.things.save(&lt;/p&gt;
{x:1, i: i}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;2) Mongostat is running and showing about 8000 inserts/sec&lt;br/&gt;
&amp;#8212;&lt;br/&gt;
insert  query update delete getmore command flushes mapped  vsize&lt;br/&gt;
res locked % idx miss %     qr|qw   ar|aw  netIn netOut  conn&lt;br/&gt;
time&lt;br/&gt;
  7818      0      0      0       0       5       0  1.95g  2.01g&lt;br/&gt;
908m     13.1          0       0|0     0|0   594k     1k     3&lt;br/&gt;
19:46:55&lt;br/&gt;
  8607      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
909m     14.7          0       0|0     0|0   654k   995b     3&lt;br/&gt;
19:46:56&lt;br/&gt;
  7616      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
910m     12.5          0       0|0     0|0   578k   995b     3&lt;br/&gt;
19:46:57&lt;br/&gt;
  7869      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
911m     13.1          0       0|0     0|0   598k   995b     3&lt;br/&gt;
19:46:59&lt;br/&gt;
  8521      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
912m       15          0       0|0     0|0   647k   995b     3&lt;br/&gt;
19:47:00&lt;br/&gt;
  7764      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
912m     13.5          0       0|0     0|0   590k   995b     3&lt;br/&gt;
19:47:01&lt;br/&gt;
  7820      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
913m     13.5          0       0|0     0|0   594k   995b     3&lt;br/&gt;
19:47:02&lt;br/&gt;
&amp;#8212;&lt;br/&gt;
I then run this command on another console talking to the same mongod:&lt;br/&gt;
&amp;gt; db.things.ensureIndex(&lt;/p&gt;
{c:1}
&lt;p&gt;, &lt;/p&gt;
{background:1}
&lt;p&gt;);&lt;/p&gt;

&lt;p&gt;It does not matter what I am indexing (the &apos;c&apos; field or &apos;a&apos; or &apos;x&apos;&lt;br/&gt;
etc).&lt;/p&gt;

&lt;p&gt;Mongostat then shows a clear increase in inserts/sec:&lt;br/&gt;
&amp;#8212;&lt;br/&gt;
  9032      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
914m     26.3          0       0|1     0|2   686k   995b     3&lt;br/&gt;
19:47:03&lt;br/&gt;
 12033      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
915m       67          0       0|1     0|2   914k   995b     3&lt;br/&gt;
19:47:04&lt;br/&gt;
 11874      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
916m     71.8          0       0|1     0|2   902k   995b     3&lt;br/&gt;
19:47:05&lt;br/&gt;
insert  query update delete getmore command flushes mapped  vsize&lt;br/&gt;
res locked % idx miss %     qr|qw   ar|aw  netIn netOut  conn&lt;br/&gt;
time&lt;br/&gt;
 10296      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
917m     82.9          0       0|1     0|2   782k   995b     3&lt;br/&gt;
19:47:06&lt;br/&gt;
 10084      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
918m     82.9          0       0|0     0|1   766k   995b     3&lt;br/&gt;
19:47:07&lt;br/&gt;
  9263      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
919m     78.1          0       0|1     0|2   704k   995b     3&lt;br/&gt;
19:47:08&lt;br/&gt;
 10551      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
920m     81.1          0       0|1     0|2   801k   995b     3&lt;br/&gt;
19:47:09&lt;br/&gt;
  8901      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
921m     80.2          0       0|1     0|2   676k   995b     3&lt;br/&gt;
19:47:10&lt;br/&gt;
 11607      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
922m     72.4          0       0|1     0|2   882k   995b     3&lt;br/&gt;
19:47:11&lt;br/&gt;
 13200      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
923m     72.9          0       0|1     0|2     1m   995b     3&lt;br/&gt;
19:47:12&lt;br/&gt;
 12968      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
924m     72.8          0       0|1     0|2   985k   995b     3&lt;br/&gt;
19:47:13&lt;br/&gt;
 13302      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
924m     73.1          0       0|1     0|2     1m   995b     3&lt;br/&gt;
19:47:14&lt;br/&gt;
 13332      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
925m     71.7          0       0|1     0|2     1m   995b     3&lt;br/&gt;
19:47:15&lt;br/&gt;
insert  query update delete getmore command flushes mapped  vsize&lt;br/&gt;
res locked % idx miss %     qr|qw   ar|aw  netIn netOut  conn&lt;br/&gt;
time&lt;br/&gt;
 13565      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
926m     72.3          0       0|0     0|1     1m   995b     3&lt;br/&gt;
19:47:16&lt;/p&gt;

&lt;p&gt;... snip excess stats ...&lt;/p&gt;

&lt;p&gt; 13153      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
991m     83.9          0       0|1     0|2   999k   995b     3&lt;br/&gt;
19:47:59&lt;br/&gt;
 12038      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
992m     86.6          0       0|1     0|2   914k   995b     3&lt;br/&gt;
19:48:00&lt;br/&gt;
 10545      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
993m     84.5          0       0|1     0|2   801k   995b     3&lt;br/&gt;
19:48:01&lt;br/&gt;
  7651      0      0      0       0       4       0  1.95g  2.01g&lt;br/&gt;
993m     19.5          0       0|0     0|0   581k     1k     3&lt;br/&gt;
19:48:02&lt;br/&gt;
  7782      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
994m     15.6          0       0|0     0|0   591k   995b     3&lt;br/&gt;
19:48:03&lt;br/&gt;
  7622      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
995m     17.4          0       0|0     0|0   579k   995b     3&lt;br/&gt;
19:48:04&lt;br/&gt;
  7670      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
995m     16.3          0       0|0     0|0   582k   995b     3&lt;br/&gt;
19:48:05&lt;br/&gt;
  8226      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
996m     19.8          0       0|0     0|0   625k   995b     3&lt;br/&gt;
19:48:06&lt;br/&gt;
insert  query update delete getmore command flushes mapped  vsize&lt;br/&gt;
res locked % idx miss %     qr|qw   ar|aw  netIn netOut  conn&lt;br/&gt;
time&lt;br/&gt;
  8019      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
997m     18.7          0       0|0     0|0   609k   995b     3&lt;br/&gt;
19:48:07&lt;br/&gt;
  7358      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
997m     16.2          0       0|0     0|0   559k   995b     3&lt;br/&gt;
19:48:08&lt;br/&gt;
  7194      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
998m     18.3          0       0|0     0|0   546k   995b     3&lt;br/&gt;
19:48:09&lt;br/&gt;
  7760      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
999m     12.6          0       0|0     0|0   589k   995b     3&lt;br/&gt;
19:48:10&lt;br/&gt;
  7586      0      0      0       0       1       0  1.95g  2.01g&lt;br/&gt;
999m     17.1          0       0|0     0|0   576k   995b     3&lt;br/&gt;
19:48:11&lt;br/&gt;
&amp;#8212;&lt;/p&gt;

&lt;p&gt;3) Finally the inserts/sec drop back to what they were when the&lt;br/&gt;
background index operation finishes.&lt;/p&gt;

&lt;p&gt;Not sure if this is a real bug or problem, just found it interesting.&lt;/p&gt;</description>
                <environment>Windows 7 64bit, Intel i7 CPU 8 Core, 8GB RAM</environment>
        <key id="14818">SERVER-2593</key>
            <summary>Background index creation inflates concurrent inserts/s operations in mongostat</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="4" iconUrl="https://jira.mongodb.org/images/icons/priorities/minor.svg">Minor - P4</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="robert@mongodb.com">Robert Stam</assignee>
                                    <reporter username="plasma">Andrew Armstrong</reporter>
                        <labels>
                    </labels>
                <created>Sat, 19 Feb 2011 03:15:16 +0000</created>
                <updated>Fri, 30 Mar 2012 14:37:03 +0000</updated>
                            <resolved>Sat, 19 Feb 2011 21:25:05 +0000</resolved>
                                    <version>1.6.5</version>
                    <version>1.7.6</version>
                                                    <component>Tools</component>
                    <component>Usability</component>
                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="24314" author="plasma" created="Sun, 20 Feb 2011 01:34:05 +0000"  >&lt;p&gt;I start with an empty collection and then after waiting for the for loop ^ to run (or this: while(true) db.things.save(&lt;/p&gt;
{x:1}
&lt;p&gt;)&lt;img class=&quot;emoticon&quot; src=&quot;https://jira.mongodb.org/images/icons/emoticons/wink.png&quot; height=&quot;16&quot; width=&quot;16&quot; align=&quot;absmiddle&quot; alt=&quot;&quot; border=&quot;0&quot;/&gt; I wait a while until the index operation takes a few seconds (so I can see the results in mongostat).&lt;/p&gt;

&lt;p&gt;That sounds quite plausible, I have plenty of free RAM.&lt;/p&gt;</comment>
                            <comment id="24313" author="eliot" created="Sun, 20 Feb 2011 01:29:00 +0000"  >&lt;p&gt;How big is the data set when you start to add index?&lt;br/&gt;
The only thing that makes sense is that the OS is keeping more of the index hot causing fewer page faults&lt;br/&gt;
From your mongostat, it does look like that could be the issue.&lt;/p&gt;

&lt;p&gt;To be more specific.&lt;/p&gt;

&lt;p&gt;Your&apos;e inserting data and index.&lt;br/&gt;
As you add, the data and index gets paged out evenly.&lt;br/&gt;
With more inserts, some index pages have to get loaded back in.&lt;br/&gt;
If the background index is happening, the index is hotter, causing it to be more sticky in ram, forcing less data in ram.&lt;br/&gt;
This makes insertions faster since there are fewer faults accessing b-tree&lt;/p&gt;</comment>
                            <comment id="24310" author="plasma" created="Sun, 20 Feb 2011 01:21:28 +0000"  >&lt;p&gt;I gave this another go this morning and even created a new database (using --dbpath), same result.&lt;/p&gt;

&lt;p&gt;I noticed that if I assign just a single CPU core to mongod, then the inserts/s decreases during background indexing as I would have expected to see.&lt;/p&gt;

&lt;p&gt;If I let it use a lot more cores, the problem appears.&lt;/p&gt;</comment>
                            <comment id="24301" author="rstam" created="Sat, 19 Feb 2011 22:00:21 +0000"  >&lt;p&gt;FYI, I am testing query performance in a different scenario, and I am loading a collection with just over 5 million documents. I was inserting at the rate of about 17000/second until I started a background index creation, at which point the insert rate dropped to about 15000/second. This makes more sense, that the rate should drop slightly. One difference is that in my new scenario I have a more complicated document.&lt;/p&gt;</comment>
                            <comment id="24296" author="eliot" created="Sat, 19 Feb 2011 21:25:05 +0000"  >&lt;p&gt;Ok.&lt;/p&gt;

&lt;p&gt;@andrew, can you try this again and see if you get the same results?  &lt;br/&gt;
if so, please let us know and give us as much detail about the system as you possibly can&lt;/p&gt;</comment>
                            <comment id="24295" author="rstam" created="Sat, 19 Feb 2011 21:22:59 +0000"  >&lt;p&gt;First I started off the insert loop.&lt;/p&gt;

&lt;p&gt;After it had inserted about 500,000 documents I issued the ensureIndex command from another window, so at that point the loop and the background index build were running concurrently.&lt;/p&gt;</comment>
                            <comment id="24293" author="eliot" created="Sat, 19 Feb 2011 21:18:14 +0000"  >&lt;p&gt;@robert, did you try it concurrently?&lt;/p&gt;</comment>
                            <comment id="24292" author="rstam" created="Sat, 19 Feb 2011 21:06:04 +0000"  >&lt;p&gt;On a Windows 7 64-bit machine with 4GB RAM and a 1TB hard drive I get about 10-11K inserts per second.&lt;/p&gt;

&lt;p&gt;I saw essentially NO change in the speed after the ensureIndex command was run.&lt;/p&gt;</comment>
                            <comment id="24287" author="scotthernandez" created="Sat, 19 Feb 2011 20:10:56 +0000"  >&lt;p&gt;As noted from the groups message, I was not able to reproduce this with 1.7.6.&lt;/p&gt;

&lt;p&gt;My machine is considerable less powerful, if that matters.&lt;/p&gt;</comment>
                            <comment id="24282" author="eliot" created="Sat, 19 Feb 2011 04:59:16 +0000"  >&lt;p&gt;@robert, can you try this on windows&lt;/p&gt;</comment>
                            <comment id="24280" author="plasma" created="Sat, 19 Feb 2011 03:17:09 +0000"  >&lt;p&gt;Also this is just a single server running, there is no replica sets.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>11.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Sat, 19 Feb 2011 04:59:16 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        13 years, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            13 years, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10026"><![CDATA[ALL]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>plasma</customfieldvalue>
            <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>robert@mongodb.com</customfieldvalue>
            <customfieldvalue>scotthernandez</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp5nr:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hridhb:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20836</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hs4nsn:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>