[SERVER-38771] Too many connections created from mongos to shards while building index on large table Created: 26/Dec/18  Updated: 27/Oct/23  Resolved: 26/Dec/18

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Chengcheng Ma Assignee: Dmitry Agranat
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File tmp_shard1.log    
Operating System: ALL
Participants:

 Description   

We'v set up sharded cluster with replica set for each shard in our production ENV. Last night we encountered a issue while connecting to the cluster and issuing building index command through mongo shell , by connecting to the mongos instance , not directly to specific shard.

The issue is : once starting building the index, connections created from mongos to this shard increases rapidly and 'too many connections' errors show up in the primary shard's log file very soon. 

From the three mongos' log, all the connections are initiated from mongos. We have googled and find related issue link as : https://jira.mongodb.org/browse/SERVER-28822 But there is no trigger conditions. And the same time, I tried to reproduce the question in test ENV ,but not occurred again. So, please help.

 

The attachment is the log details while mongod encountered the error.

 



 Comments   
Comment by Chengcheng Ma [ 27/Dec/18 ]

Sorry, mysql mistakes.

That operation was issued by a mongorestore command, so the index creation would be foreground, not background. 

Sorry to disturb.

Comment by Dmitry Agranat [ 26/Dec/18 ]

Hi cora_ma,

The behavior you've described is expected.

By default, creating an index on a populated collection blocks all other operations on a database. When building an index on a populated collection, the database that holds the collection is unavailable for read or write operations until the index build completes. Any operation that requires a read or write lock on all databases will wait for the foreground index build to complete.

For a replica set or a sharded cluster, use a rolling index build. See Build Indexes on Replica Sets for details.

Based on the above, I do not see anything to indicate a bug in the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-user group.

See also our Technical Support page for additional support resources.

Thank you,
Dima

Comment by Chengcheng Ma [ 26/Dec/18 ]

The version of mongodb we are using is 3.4.18.

And the issue also occurs in 3.2.20.

Generated at Thu Feb 08 04:49:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.