[SERVER-8132] slowness in mongos when dealing with replset due to getServerAddress() calls Created: 10/Jan/13  Updated: 11/Jul/16  Resolved: 15/Jan/13

Status: Closed
Project: Core Server
Component/s: Performance
Affects Version/s: 2.2.2
Fix Version/s: 2.2.3, 2.4.0-rc0

Type: Bug Priority: Major - P3
Reporter: Daniel Pasette (Inactive) Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File cacheAddr.diff     PDF File gprof-slowreplset.pdf     PDF File res2.pdf     PDF File res3.pdf    
Operating System: ALL
Steps To Reproduce:

This can be seen easily when running a high performance test (1000s of rps) with multiple threads (10+) and comparing the performance between replset and master/slave.

Participants:

 Description   

When doing high performance benchmark with a 2 shard cluster, throughput was 2.5x higher when using master/slave rather than replset.
This is obviously a huge performance degradation.
From profiling it seems due to the many calls to DBClientReplicaSet.getServerAddress() which take a lock.
By applying the tiny provided patch that caches the value, the issue goes away.



 Comments   
Comment by auto [ 23/Jan/13 ]

Author:

{u'date': u'2013-01-22T23:31:46Z', u'email': u'dan@10gen.com', u'name': u'Dan Pasette'}

Message: SERVER-8132: removed unordered_* and re-ordered imports in hostandport.h
to enable backport to 2.2.
Branch: v2.2
https://github.com/mongodb/mongo/commit/0adb9623539b7cf8bc9bb32bb61b9b83db9c0fda

Comment by auto [ 23/Jan/13 ]

Author:

{u'date': u'2013-01-15T06:06:58Z', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: SERVER-8132: fix caching of writeback init
Branch: v2.2
https://github.com/mongodb/mongo/commit/31150cee4cf5aae5fa54ab8d34a8f8452b11b0f1

Comment by auto [ 23/Jan/13 ]

Author:

{u'date': u'2013-01-15T02:05:02Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-8132: speed up getServerAddress
Branch: v2.2
https://github.com/mongodb/mongo/commit/109f6fbde0f802413fa80b699a2e2d39cf9b32fc

Comment by Antoine Girbal [ 21/Jan/13 ]

Tested the fix with nightly build from 1/20, and it's all good.

Comment by auto [ 15/Jan/13 ]

Author:

{u'date': u'2013-01-15T06:06:58Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-8132: fix caching of writeback init
Branch: master
https://github.com/mongodb/mongo/commit/69e3bd18b5f82a727cf3dcd5e472cbe07d40162c

Comment by auto [ 15/Jan/13 ]

Author:

{u'date': u'2013-01-15T02:05:02Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}

Message: SERVER-8132: speed up getServerAddress
Branch: master
https://github.com/mongodb/mongo/commit/f5cc553ffa9a8c61b96f6ac0cdb5c7ec08dfd8ce

Comment by Antoine Girbal [ 10/Jan/13 ]

patch to add caching

Comment by Antoine Girbal [ 10/Jan/13 ]

patched mongos profile with replset

Comment by Antoine Girbal [ 10/Jan/13 ]

mongos profile with --master

Comment by Antoine Girbal [ 10/Jan/13 ]

mongos profile with replset

Generated at Thu Feb 08 03:16:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.