[SERVER-56147] In connection PoolForHost, a thread blocked waiting for free connection may be never notified Created: 16/Apr/21  Updated: 29/Oct/23  Resolved: 10/May/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.20
Fix Version/s: 4.0 Required

Type: Bug Priority: Major - P3
Reporter: Andrew Shuvalov (Inactive) Assignee: Andrew Shuvalov (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Problem/Incident
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:
Linked BF Score: 184

 Description   

waitForFreeConnection() is blocked for condition variable here. To work properly, the _cv should be notified each time the `_checkedOut` is decremented.

1. In one code path, in PoolForHost::done() `_checkedOut` is properly followed by notify, because the only call site is DBConnectionPool::release().

Even though this is correct, it is over-optimization. It is clearer to notify near the place where condition is changed. If this is regarded as an important optimization, a comment should be added.

2. However in DBConnectionPool::decrementEgress() the `_checkedOut` is decremented and the _cv is not notified. The call site is ScopedDbConnection::kill(). This must be fixed.

When using ScopedDbConnection, its deletion will not trigger reusing the connection.

A possible deadlock created by this bug may hit the ReplicaSetMonitor. In Refresher::_refreshUntilMatches() we are using the ScopedDbConnection here. If ReplicaSetMonitor is releasing a scoped connection, this will not trigger anyone waiting for it. If that was the moment RSM detected a change, the deadlock may prevent other blocked threads to take advantage of freed connection.

This problem may not be present in v4.2 and late4r because of significant refactoring like SERVER-35688 that were impossible to backport.



 Comments   
Comment by Githook User [ 10/May/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56147: In connection PoolForHost, a thread blocked waiting for free connection may be never notified
Branch: v4.0
https://github.com/mongodb/mongo/commit/fa92dae8174f68fec1018d572f14e490e5467eac

Comment by Githook User [ 07/May/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56147: revert connection pool notification fix with test crashing on Windows
Branch: v4.0
https://github.com/mongodb/mongo/commit/23334d09cd768898b6524cfe6499d7d036f20d21

Comment by Githook User [ 26/Apr/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56217 SERVER-56147: maxed out connection pool should respect get() timeout and CV should be notified on connection release
Branch: v4.0
https://github.com/mongodb/mongo/commit/3d8730be2c747dfe869614e215808ceddafa8516

Comment by Githook User [ 26/Apr/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56217 SERVER-56147: maxed out connection pool should respect get() timeout and CV should be notified on connection release
Branch: v4.0
https://github.com/mongodb/mongo/commit/1d5cbc113d4318b79831113790c3abd09ba25098

Comment by Githook User [ 26/Apr/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56217 SERVER-56147: maxed out connection pool should respect get() timeout and CV should be notified on connection release
Branch: v4.0
https://github.com/mongodb/mongo/commit/5c8ec91022cd185490fb960a5a7fdaf1c58f94d7

Comment by Githook User [ 23/Apr/21 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-56147: In connection PoolForHost, a thread blocked waiting for free connection may be never notified
Branch: v4.0
https://github.com/mongodb/mongo/commit/7a93b8e33051ea6e0b65f7431b736f71091509f8

Generated at Thu Feb 08 05:38:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.