[SERVER-2599] potential connection leak in sharded environment Created: 21/Feb/11 Updated: 17/Mar/11 Resolved: 21/Feb/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | 1.6.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Benedikt Waldvogel | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
|
||
| Attachments: |
|
| Operating System: | Linux |
| Participants: |
| Description |
|
I have a system setup that is very similar to the architecture described in http://www.snailinaturtleneck.com/blog/2010/08/30/return-of-the-mongo-mailbag/ So here are a few weird observations that I made: The clients use the Java driver in version 2.4 and the client instance is kept open and reused for each query. |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 28/Feb/11 ] |
|
The server sets keep alive, so its not strictly required on the java side, added a case though I believe serverStatus and netstat disagree because the mongod process isn't notified in the same way the tcp/ip stack is. |
| Comment by Benedikt Waldvogel [ 28/Feb/11 ] |
|
That's true and I agree that it's a bad behavior of the firewall. |
| Comment by Eliot Horowitz (Inactive) [ 25/Feb/11 ] |
|
Keep alive is set. Default kernel keep alive settings kick in at 2 hours. Routers aren't supposed to kill idle connections for 2 hours in general. |
| Comment by Benedikt Waldvogel [ 25/Feb/11 ] |
|
Turns out that the frequent connection timeouts happen because the firewall silently drops connections that are idle for at least one hour. |
| Comment by Benedikt Waldvogel [ 23/Feb/11 ] |
|
I've attached the logfile. |
| Comment by Eliot Horowitz (Inactive) [ 22/Feb/11 ] |
|
Can you attach the mongos log? |
| Comment by Benedikt Waldvogel [ 22/Feb/11 ] |
|
https://gist.github.com/838638#file_mongod_netstat.txt |
| Comment by Eliot Horowitz (Inactive) [ 21/Feb/11 ] |
|
Can you show serverStatus(), connPoolStats and netstat from the same point in time? |
| Comment by Benedikt Waldvogel [ 21/Feb/11 ] |
|
how do you explain that db.serverStatus() reports over 100 connections while netstat shows only 50? |
| Comment by Eliot Horowitz (Inactive) [ 21/Feb/11 ] |
|
mongos keeps connections open to all shards all the time. You can run To see The writeback connections are also supposed to be there, and live forever. |