[SERVER-9816] inconsistent connection count Created: 30/May/13  Updated: 10/Dec/14  Resolved: 31/May/13

Status: Closed
Project: Core Server
Component/s: Diagnostics
Affects Version/s: 2.2.2
Fix Version/s: None

Type: Question Priority: Minor - P4
Reporter: sorghum Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: connection
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Red Hat Enterprise Linux Server release 6.3 (Santiago)


Participants:

 Description   

Executing "mongostat --port 27017" shows the number of connections is 15K,but running "netstat -na | wc -l" to count the connections, the result is only 95.
Why are the two results inconsistent?



 Comments   
Comment by Scott Hernandez (Inactive) [ 31/May/13 ]

I'm glad to hear your production system is stable again.

When you setup that test system and create a new issue if you find useful information please make sure to include client details and your connection options (esp. if you set a socket/read timeout as those settings can dramatically affect connections). Also if you have private data please feel free to create a Community Private (SUPPORT) issue so that it is not public.

Linking this issue with the new one can be done after you create the new issue by you, or us.

Comment by sorghum [ 31/May/13 ]

db.serverStatus() shows the value of connections.current is closed to the number given by mongostat.
netstat -na illustrates the output like follows:

tcp 0 0 0.0.0.0:27018 0.0.0.0:* LISTEN
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:50249 ESTABLISHED
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:49775 ESTABLISHED
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:33696 ESTABLISHED
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:34812 ESTABLISHED
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:57724 ESTABLISHED
tcp 0 0 xx.xx.xx.xx:27018 yy.yy.yy.yy:57991 ESTABLISHED

-------------------------------

It's a production enviroment, and it works fine now. So I cannot give you the result of currentOp and serverStatus.
I will setup a test cluster to reproduce this problem, and give you more details.

Right now, I doubt the reason is the "locked db". When the locked percentage is low, the results are consistent. When the percentage is extremely high, the results differ greatly with each other.

Comment by Scott Hernandez (Inactive) [ 31/May/13 ]

On the primary can you run

db.currentOp(true);
db.serverStatus()

and the netstat -na so we can see all the numbers.

If you want this info to be private we can move this issue someplace less public.

Comment by sorghum [ 31/May/13 ]

The architecture of mongodb is a replicaset cluster with 3 nodes (primary, secondary,and arbiter)
The following are the output of the two commands:

Primary:

[root@mongo-online01 ~]# netstat -nat | grep xx.xx.xx.xx:27018 | wc -l
649
[root@mongo-online01 ~]# mongostat --port 27018 -u root -p root
connected to: 127.0.0.1:27018
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn set repl time
10 0 1 1 22 70 0 26.1g 70.6g 1.54g 3 my_collection:111.0% 0 17401|0 1|0 9k 25k 17463 root PRI 09:29:40
8 1 1 6 14 130 0 26.1g 70.6g 1.54g 1 my_collection:107.2% 0 17400|0 0|1 13k 38k 17461 root PRI 09:29:41
6 1 1 7 12 135 0 26.1g 70.6g 1.62g 2 my_collection:100.2% 0 17401|0 1|0 13k 37k 17462 root PRI 09:29:42
3 2 1 5 8 88 0 26.1g 70.6g 1.43g 4 my_collection:85.0% 0 17404|0 1|0 9k 25k 17465 root PRI 09:29:43

--------------------------------------------------------------------------------------------------------------------------------------------------

Secondary:

[root@mongo-online02 ~]# netstat -nat | grep yy.yy.yy.yy:27018 | wc -l
74
[root@mongo-online02 ~]# mongostat --port 27018 -u root -p root
connected to: 127.0.0.1:27018
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn set repl time
*8 18 *0 *0 0 68|0 0 24.3g 49.3g 238m 2 my_collection:1.5% 0 25|0 12|0 8k 31k 75 root SEC 09:29:00
*7 13 *0 *0 0 54|0 0 24.3g 49.3g 232m 2 my_collection:1.4% 0 5|0 32|0 6k 20k 74 root SEC 09:29:01
*5 17 *0 *0 0 53|0 0 24.3g 49.3g 302m 1 my_collection:0.2% 0 13|0 23|0 6k 21k 74 root SEC 09:29:02

Comment by Scott Hernandez (Inactive) [ 30/May/13 ]

Please provide the output of the raw commands. This will allow us to see the current state of the connections reported by netstat and mongostat numbers.

Also, please make sure that the connections you are counting with netstat are to port 27017 and not other things.

Generated at Thu Feb 08 03:21:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.