[CDRIVER-2389] Deprecate minPoolSize and log a warning if it's used Created: 17/Nov/17  Updated: 28/Oct/23  Resolved: 05/Dec/17

Status: Closed
Project: C Driver
Component/s: libmongoc, network
Affects Version/s: None
Fix Version/s: 1.9.0

Type: Improvement Priority: Major - P3
Reporter: Rustam Abdullaev Assignee: A. Jesse Jiryu Davis
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 10 x64


Issue Links:
Related
related to CDRIVER-4663 Eagerly create minPoolSize connections In Code Review
related to CDRIVER-2131 mongoc_client_pool_push should mentio... Closed
is related to CDRIVER-1558 minPoolSize bugs Closed
is related to CDRIVER-2390 Remove minPoolSize option Backlog
Case:

 Description   

Summary: minPoolSize does not do what anyone expects, and it's inconsistent with other drivers' options that have the same or similar names. What it actually does is not useful. Deprecate it, remove it from the main mongoc_uri_t documentation, and log a warning if it's used.

Original report: Setting minPoolSize in the URL to a non-zero value causes frequent connects/disconnects in a threaded environment. That is not the behavior that one would expect from this setting.

Steps to reproduce:

#include <iostream>
#include <thread>
#include <vector>
#include <bsoncxx/builder/stream/document.hpp>
#include <mongocxx/instance.hpp>
#include <mongocxx/client.hpp>
#include <mongocxx/pool.hpp>
 
using namespace bsoncxx::builder::stream;
using namespace std;
 
int main() {
  mongocxx::instance inst;
  mongocxx::uri uri("mongodb://localhost/test?minPoolSize=1");
  mongocxx::pool pool(uri);
  pool.acquire()->database("test").collection("test").drop();
  vector<thread> thr;
  for (int j = 0; j < 3; ++j) {
    thr.emplace_back([&pool] {
      for (int i = 0; i < 2000; ++i) {
        auto conn = pool.acquire();
        auto db = conn->database("test");
        db.collection("test").insert_one(document{} << "idx" << i << finalize);
      }
    });
  }
  for (auto& t : thr) {
    if (t.joinable())
      t.join();
  }
}

Results:

minPoolSize=0

2017-11-17T09:40:25.295+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54264 #1 (1 connection now open)
2017-11-17T09:40:25.296+0100 I NETWORK  [conn1] received client metadata from 127.0.0.1:54264 conn1: { driver: { name: "mongoc / mongocxx", version: "1.7.0-dev / 3.1.1-pre" }, os: { type: "Windows", name: "Windows", version: "6.2 (9200)", architecture: "x86_64" }, platform: "cfg=0x200c9 CC=MSVC 1900 CFLAGS=" /DWIN32 /D_WINDOWS /W3" LDFLAGS=" /machine:x64"" }
2017-11-17T09:40:25.297+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54265 #2 (2 connections now open)
2017-11-17T09:40:25.306+0100 I COMMAND  [conn2] CMD: drop test.test
2017-11-17T09:40:25.311+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54266 #3 (3 connections now open)
2017-11-17T09:40:25.311+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54267 #4 (4 connections now open)
2017-11-17T09:40:25.608+0100 I -        [conn3] end connection 127.0.0.1:54266 (4 connections now open)
2017-11-17T09:40:25.608+0100 I -        [conn2] end connection 127.0.0.1:54265 (4 connections now open)
2017-11-17T09:40:25.608+0100 I -        [conn4] end connection 127.0.0.1:54267 (4 connections now open)
2017-11-17T09:40:25.608+0100 I -        [conn1] end connection 127.0.0.1:54264 (2 connections now open)

minPoolSize=1

2017-11-17T09:43:34.750+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54237 #1 (1 connection now open)
2017-11-17T09:43:34.761+0100 I NETWORK  [conn1] received client metadata from 127.0.0.1:54237 conn1: { driver: { name: "mongoc / mongocxx", version: "1.7.0-dev / 3.1.1-pre" }, os: { type: "Windows", name: "Windows", version: "6.2 (9200)", architecture: "x86_64" }, platform: "cfg=0x200c9 CC=MSVC 1900 CFLAGS=" /DWIN32 /D_WINDOWS /W3" LDFLAGS=" /machine:x64"" }
2017-11-17T09:43:34.762+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54238 #2 (2 connections now open)
2017-11-17T09:43:34.763+0100 I COMMAND  [conn2] CMD: drop test.test
2017-11-17T09:43:34.769+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54239 #3 (3 connections now open)
2017-11-17T09:43:34.769+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54240 #4 (4 connections now open)
2017-11-17T09:43:34.820+0100 I -        [conn4] end connection 127.0.0.1:54240 (4 connections now open)
2017-11-17T09:43:34.820+0100 I -        [conn2] end connection 127.0.0.1:54238 (4 connections now open)
2017-11-17T09:43:34.822+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54241 #5 (3 connections now open)
2017-11-17T09:43:34.822+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54242 #6 (4 connections now open)
2017-11-17T09:43:34.873+0100 I -        [conn6] end connection 127.0.0.1:54242 (4 connections now open)
2017-11-17T09:43:34.874+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54243 #7 (4 connections now open)
2017-11-17T09:43:34.874+0100 I -        [conn5] end connection 127.0.0.1:54241 (3 connections now open)
2017-11-17T09:43:34.874+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54244 #8 (4 connections now open)
2017-11-17T09:43:34.880+0100 I -        [conn3] end connection 127.0.0.1:54239 (4 connections now open)
2017-11-17T09:43:34.880+0100 I -        [conn8] end connection 127.0.0.1:54244 (4 connections now open)
2017-11-17T09:43:34.881+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54245 #9 (3 connections now open)
2017-11-17T09:43:34.881+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54246 #10 (4 connections now open)
2017-11-17T09:43:34.892+0100 I -        [conn7] end connection 127.0.0.1:54243 (4 connections now open)
2017-11-17T09:43:34.892+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54247 #11 (4 connections now open)
2017-11-17T09:43:34.974+0100 I -        [conn9] end connection 127.0.0.1:54245 (4 connections now open)
2017-11-17T09:43:34.975+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54248 #12 (4 connections now open)
2017-11-17T09:43:35.061+0100 I -        [conn10] end connection 127.0.0.1:54246 (4 connections now open)
2017-11-17T09:43:35.062+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54249 #13 (4 connections now open)
2017-11-17T09:43:35.083+0100 I -        [conn12] end connection 127.0.0.1:54248 (4 connections now open)
2017-11-17T09:43:35.083+0100 I NETWORK  [thread1] connection accepted from 127.0.0.1:54250 #14 (4 connections now open)
2017-11-17T09:43:35.091+0100 I -        [conn13] end connection 127.0.0.1:54249 (4 connections now open)
2017-11-17T09:43:35.110+0100 I -        [conn11] end connection 127.0.0.1:54247 (3 connections now open)
2017-11-17T09:43:35.130+0100 I -        [conn14] end connection 127.0.0.1:54250 (2 connections now open)
2017-11-17T09:43:35.130+0100 I -        [conn1] end connection 127.0.0.1:54237 (1 connection now open)



 Comments   
Comment by Githook User [ 05/Dec/17 ]

Author:

{'username': 'ajdavis', 'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis'}

Message: CDRIVER-2389 deprecate minpoolsize
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/f4d0da831eea29b66015ba25ed119cb9b0ab29d5

Comment by Rustam Abdullaev [ 18/Nov/17 ]

Not what anyone expects indeed; it doesn't work as in PyMongo, it doesn't work as documented in mongodb reference and I can't think of any use for what it actually does. I think at least the delay before new connections are established should be increased.

Meanwhile we're going to have to add a sanitizer for the URL in our code because this isn't the first time someone tries to enable it in production :\

Comment by A. Jesse Jiryu Davis [ 18/Nov/17 ]

This is a C Driver design flaw. The minPoolSize option works as documented, which unfortunately is probably not what anyone expects:

http://mongoc.org/libmongoc/current/mongoc_uri_t.html#connection-pool-options

minPoolSize is "the number of clients to keep in the pool; once it is reached, mongoc_client_pool_push() destroys clients instead of pushing them." It could be better named "maxIdleClients". It is not a very useful setting. It does not work the same as minPoolSize in PyMongo or minConnectionsPerHost in the Java Driver, which more likely match your expectations. Those options maintain at least minPoolSize open connections, eagerly opening connections even if they are not yet demanded.

I haven't come up with a backwards-compatible strategy for fixing this problem. Probably we should deprecate the option, log a warning whenever it's configured, and not introduce a new option that matches PyMongo's minPoolSize behavior unless there is a very strong desire for it.

Meanwhile, please don't use minPoolSize at all.

Generated at Wed Feb 07 21:15:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.