-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Unknown
-
Affects Version/s: None
-
Component/s: None
-
None
-
(copied to CRM)
-
Not Needed
-
None
-
C Drivers
-
Not Needed
-
-
None
-
None
-
None
-
None
-
None
-
None
Proposal: When a client is pushed back onto the pool, reduce the size of large connection buffers.
Background & Motivation
HELP-78501 reports observed memory growth in a long-lived process. This appears to be due to reading large responses on a buffered stream. The long-lived buffered stream does not later reduce the size until the pool is destroyed.
An example creating many cursors reading large responses reproduces the issue. Here is a simplified snippet:
mongoc_client_t *client[100]; for (int i = 0; i < 100; i++) { client[i] = mongoc_client_pool_pop (pool); } for (int i = 0; i < 100; i++) { // Server replies may be 16MB each. find_all_documents (client[i], "db", "big_collection"); } // 1600 MB allocated and will not be freed until `mongoc_client_pool_destroy`.
macOS Instruments shows many 16MB buffers allocated after calls to find_all_documents complete:
After calling mongoc_client_pool_destroy, memory usage lowers to near 0.
The default stream initiator creates buffered streams.