[SERVER-1714] multiplex mongos -> mongod connecions after successful read op Created: 01/Sep/10  Updated: 12/Jul/16  Resolved: 08/Jan/14

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 2.5.5

Type: Improvement Priority: Major - P3
Reporter: Eliot Horowitz (Inactive) Assignee: Greg Studer
Resolution: Done Votes: 10
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-8224 when lots of web server, there is t... Closed
Duplicate
is duplicated by SERVER-9630 Mongos crashes by few seconds of DB l... Closed
is duplicated by SERVER-6402 Remove WriteBackListener entirely, re... Closed
Related
related to DOCS-2947 Document multiplex improvement in 2.6... Closed
related to SERVER-9040 Cap number of connections between mon... Closed
is related to SERVER-4466 Make mongos connection pool more conf... Closed
is related to SERVER-7606 Refactor how shard version informatio... Closed
is related to SERVER-9022 Enable mongos mode which releases Sha... Closed
Participants:

 Description   

Can't do this until state moves from per connection to per reqeust.



 Comments   
Comment by Grégoire Seux [ 28/Feb/13 ]

We have exactly the same issue:
14 shards, around 10k connections per primary. A lot of ram is lost with this.
At some point mongodb used one thread per connection, if this is still the case, we have 10k threads per primary and nobody thinks this is a good situation ^^

Comment by Kevin J. Rice [ 27/Feb/13 ]

We're seeing (with 48 shards in our cluster, 24 primaries on each of two boxes, primary/replica/arbiter setup w/ 3 configs), upwards of 600+ connections per primary mongod process. This is huge. It prevents us from going to rather large numbers of shards, say 500 on one box.

Further, IIRC, if I have 10 processes connect to one mongos and that mongos connects to 4 shards, I've really instantiated 40 connections. This is huge. I am going to have 3000 servers, each with a pymongo client, connecting and sending data. This means 3000 * 4 = 12,000 connections.

I may be able to handle this with virtual IP addresses for each of the mongod's. But, it's an ugly workaround.

Comment by Eliot Horowitz (Inactive) [ 27/Jul/11 ]

with stack size changes, this is less important

Comment by auto [ 19/May/11 ]

Author:

{u'login': u'erh', u'name': u'Eliot Horowitz', u'email': u'eliot@10gen.com'}

Message: don't unset sharding state when a shard connection is done
more prep for SERVER-1714
Branch: master
https://github.com/mongodb/mongo/commit/0b689c83e8a8555d2167ef0337982008ff698c14

Comment by Eliot Horowitz (Inactive) [ 18/May/11 ]

Left:

  • fix drop hack
  • figure out repeated getLastError
  • actually turn on
  • pool stats aggregrated
Comment by Eliot Horowitz (Inactive) [ 11/Nov/10 ]

this is going to be too expensive till we can re-work network so version is per request

Generated at Thu Feb 08 02:57:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.