[SERVER-9924] writing to a sharded collection vs an unsharded collection shows a 50-60% throughput drop off Created: 13/Jun/13  Updated: 21/Jun/13  Resolved: 20/Jun/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.4.3
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Edouard Servan-Schreiber Assignee: Osmar Olivo
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

when doing inserts with a single thread to an unsharded collection in a cluster of 4 shards, we observed a throughput of ~4k inserts/sec.
When doing inserts to a sharded collection, the overall throughput dropped to ~1.8k/sec. A drop of 55%. Per shard it was a drop by a factor of 10.
When doing inserts to a hashed sharded collection, the overall throughput dropped to ~1.5k/sec.

The sharded collection was well balanced and the throughput was well balanced across all shards (~400 inserts/sec/shard).

This was done on a single machine with an SSD, 4 mongod's running as a sharded cluster with single replica shards.

What could explain such a performance drop off?



 Comments   
Comment by Osmar Olivo [ 21/Jun/13 ]

The test scripts were not written by 10gen initially and thus cannot be posted publicly by us. The negative results were a product of human error. Thanks.

Comment by vinay pothnis [ 20/Jun/13 ]

Hi there,

Could you please post the test script and elaborate on the error in the test?

Thanks!

Comment by Edouard Servan-Schreiber [ 20/Jun/13 ]

Full tests observe only a 10% slowdown between a manual sharding approach (direct loading into mongod's) and sharded approach leveraging mongos. Product works as designed.
Test featured in this ticket had an error and results were not reproduced.

Comment by Edouard Servan-Schreiber [ 13/Jun/13 ]

any news on this?

Generated at Thu Feb 08 03:21:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.