[SERVER-2593] Background index creation inflates concurrent inserts/s operations in mongostat Created: 19/Feb/11 Updated: 30/Mar/12 Resolved: 19/Feb/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Tools, Usability |
| Affects Version/s: | 1.6.5, 1.7.6 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Andrew Armstrong | Assignee: | Robert Stam |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows 7 64bit, Intel i7 CPU 8 Core, 8GB RAM |
||
| Operating System: | ALL |
| Participants: |
| Description |
|
(Copied from http://groups.google.com/group/mongodb-user/browse_thread/thread/f65c2178ade3f9c8) Hi, I was just playing with MongoDB 1.6.5 (and development 1.7.6) and I Is it really increasing or is there some bug here? 1) Start off with this running on a separate console: ); 2) Mongostat is running and showing about 8000 inserts/sec , {background:1}); It does not matter what I am indexing (the 'c' field or 'a' or 'x' Mongostat then shows a clear increase in inserts/sec: ... snip excess stats ... 13153 0 0 0 0 1 0 1.95g 2.01g 3) Finally the inserts/sec drop back to what they were when the Not sure if this is a real bug or problem, just found it interesting. |
| Comments |
| Comment by Andrew Armstrong [ 20/Feb/11 ] |
|
I start with an empty collection and then after waiting for the for loop ^ to run (or this: while(true) db.things.save( {x:1}) That sounds quite plausible, I have plenty of free RAM. |
| Comment by Eliot Horowitz (Inactive) [ 20/Feb/11 ] |
|
How big is the data set when you start to add index? To be more specific. Your'e inserting data and index. |
| Comment by Andrew Armstrong [ 20/Feb/11 ] |
|
I gave this another go this morning and even created a new database (using --dbpath), same result. I noticed that if I assign just a single CPU core to mongod, then the inserts/s decreases during background indexing as I would have expected to see. If I let it use a lot more cores, the problem appears. |
| Comment by Robert Stam [ 19/Feb/11 ] |
|
FYI, I am testing query performance in a different scenario, and I am loading a collection with just over 5 million documents. I was inserting at the rate of about 17000/second until I started a background index creation, at which point the insert rate dropped to about 15000/second. This makes more sense, that the rate should drop slightly. One difference is that in my new scenario I have a more complicated document. |
| Comment by Eliot Horowitz (Inactive) [ 19/Feb/11 ] |
|
Ok. @andrew, can you try this again and see if you get the same results? |
| Comment by Robert Stam [ 19/Feb/11 ] |
|
First I started off the insert loop. After it had inserted about 500,000 documents I issued the ensureIndex command from another window, so at that point the loop and the background index build were running concurrently. |
| Comment by Eliot Horowitz (Inactive) [ 19/Feb/11 ] |
|
@robert, did you try it concurrently? |
| Comment by Robert Stam [ 19/Feb/11 ] |
|
On a Windows 7 64-bit machine with 4GB RAM and a 1TB hard drive I get about 10-11K inserts per second. I saw essentially NO change in the speed after the ensureIndex command was run. |
| Comment by Scott Hernandez (Inactive) [ 19/Feb/11 ] |
|
As noted from the groups message, I was not able to reproduce this with 1.7.6. My machine is considerable less powerful, if that matters. |
| Comment by Eliot Horowitz (Inactive) [ 19/Feb/11 ] |
|
@robert, can you try this on windows |
| Comment by Andrew Armstrong [ 19/Feb/11 ] |
|
Also this is just a single server running, there is no replica sets. |