[SERVER-21266] Bulk loading to micro-sharded cluster slower in 3.2.0-rc1 vs 3.0.6 Created: 03/Nov/15 Updated: 09/Dec/15 Resolved: 08/Nov/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Write Ops |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | John Page | Assignee: | Misha Tyulenev |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Sprint: | Sharding C (11/20/15) |
| Participants: |
| Description |
|
Testing a bulk load into a sharded cluster - 40 microshards of wired tiger (aggreagtion cluster) on a 40 core server, loads of RAM, 8500 PIOPS, . Version 3.0.6 gets a steady 625,000 inserts/s - same hardware 3.2.0-rc1 only gets 520,000 inserts/s, that's a 17% performance drop. |
| Comments |
| Comment by John Page [ 08/Nov/15 ] |
|
Re-tested on a more controlled environment and I now see 20% better on 3.2 as well so closing. Must have been some Amazon weirdness of PEBKAC. |
| Comment by Misha Tyulenev [ 05/Nov/15 ] |
|
Cant reproduce locally, john.page could you please confirm that its reproducible and provide me with an access to the environment? I scaled down the test to match my machine: loading 37M documents instead of 290M and running 24 shards instead of 40 Here are the results 37390457 documents (/fastload test_result_2013.txt) 3.2 rc2 (359K) 3.2 rc1 (346K) 3.0.7 (283K) 3.0.6 rc2 (283K) mongos 3.0.6 rc2, mongod 3.2.rc1 (306K) |
| Comment by Andy Schwerin [ 03/Nov/15 ] |
|
Maybe it's an issue that appears as the number of shards increases. I think it's worth trying the 40 shards on a box similar to the one John used. |
| Comment by Andy Schwerin [ 03/Nov/15 ] |
|
Misha's going to try to repro on the assumption that this is a mongos problem. |
| Comment by John Page [ 03/Nov/15 ] |
|
Single config server - not sure what --configsvr does in 3.2, dont think it uses a replica |
| Comment by Andy Schwerin [ 03/Nov/15 ] |
|
SCCC config servers? Could you retest with 3.0 mongos and 3.2 mongos? I assume you'll see 3.0-ish behaviour, but want to confirm. |
| Comment by John Page [ 03/Nov/15 ] |
|
I should add - Wired Tiger not MMAP |
| Comment by Daniel Pasette (Inactive) [ 03/Nov/15 ] |
|