[SERVER-7331] Optimize mongos performance Created: 11/Oct/12  Updated: 07/Mar/14  Resolved: 07/Mar/14

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 2.2.0
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Ben Becker Assignee: Unassigned
Resolution: Won't Fix Votes: 7
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

All
Test Bed: Physical machine with CentOS 6.3, Linux kernel 2.6.32-279.9.1.el6
8 gigs of memory
8 cores
Client tests were run locally to avoid profiling network latency (as direct connections were not available for these machines).


Attachments: File mongod1_nolog.ps     File mongod3_nogetShardsForQuery.ps     File mongos1_nogetShardsForQuery.ps     File mongos1_nolog.ps     File mongos1_withlog.ps    
Issue Links:
Depends
depends on SERVER-7231 Replace log(int level) statements wit... Closed
Duplicate
duplicates SERVER-2768 mongos speed improvements Closed
Related
related to SERVER-7422 Optimize FieldRangeSet::range() Closed
Participants:

 Description   

Several performance issues have been identified in mongos. SERVER-7231 addresses one issue with logging in ParallelSortClusterCursor. There is much more room for optimization, though.

Baseline

In a simple test which floods mongos with a valid findOne() query, mongos was initially handling 20k queries per second. If the clients connected directly to the shard with the data, the shard would handle 50-60k queries per second. See mongos1_withlog.ps.

No Logging

The first set ('nolog') is based on v2.2, but with the log() statements in parallel.cpp commented out. This increased mongos' concurrent query volume to 30k queries per second. See mongos1_nolog.ps, and mongod1_nolog.ps.

Short-circuit getShardsForQuery()

The second set ('nogetShardsForQuery') includes the 'nolog' patch, and also immediately returns the first shard:

     void ChunkManager::getShardsForQuery( set<Shard>& shards , const BSONObj& query ) const {
+        shards.insert(_chunkRanges.ranges().begin()->second->getShard());
+        return;

When testing with a query that targets this shard, mongos handles 40-45k queries per second. See mongos1_nogetShardsForQuery.ps, and mongod3_nogetShardsForQuery.ps.


Generated at Thu Feb 08 03:14:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.