Details
-
Bug
-
Resolution: Works as Designed
-
Critical - P2
-
None
-
None
-
None
-
None
Description
Problem description:
in our application, the incoming requests are queued and served by a thread from a thread pool.
In this case, if we iterate over a result set with mongocxx::cursor, it seems, that cursor won't free up allocated memory and the process memory consumption will increase even that the cursor created locally and will be destroyed when it goes out of scope.
(Note: we have developed a counter example, where every requests spawn a new thread and served on it (Please note: new thread, not the same thread with reuse), we couldn't observe such a huge memory consumption in this case)
The sample code that performs a find and iteration:
vector<Result> Query() {
|
initializePoolOneTime();
|
|
|
auto client =_pool->acquire();
|
mongocxx::database db = (*client)["Database"]; |
mongocxx::collection collection = db["keyword"]; |
mongocxx::cursor cursor = collection.find(document{} << "XXX" << finalize); |
|
|
vector<Result> result;
|
for(auto&& m : cursor) { |
Result c;
|
c.chunk = getIntValue(m["YYY"]); |
chunks.push_back(c);
|
}
|
}
|
Running our application by valgrind shows the following stack trace:
==20124== 524,288 bytes in 1 blocks are still reachable in loss record 1,065 of 1,065 |
==20124== at 0x4C33D2F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) |
==20124== by 0x77C999: bson_realloc (bson-memory.c:154) |
==20124== by 0x77CA04: bson_realloc_ctx (bson-memory.c:194) |
==20124== by 0x75E162: _mongoc_buffer_fill (mongoc-buffer.c:254) |
==20124== by 0x73324D: mongoc_stream_buffered_readv (mongoc-stream-buffered.c:240) |
==20124== by 0x733D09: mongoc_stream_readv (mongoc-stream.c:237) |
==20124== by 0x733E45: mongoc_stream_read (mongoc-stream.c:281) |
==20124== by 0x75DF38: _mongoc_buffer_append_from_stream (mongoc-buffer.c:200) |
==20124== by 0x6FB71C: mongoc_cluster_run_opmsg (mongoc-cluster.c:3468) |
==20124== by 0x6F5A7B: mongoc_cluster_run_command_monitored (mongoc-cluster.c:544) |
==20124== by 0x70A27A: _mongoc_cursor_run_command (mongoc-cursor.c:1052) |
==20124== by 0x70BAA1: _mongoc_cursor_response_refresh (mongoc-cursor.c:1673) |
==20124== by 0x70D09C: _prime (mongoc-cursor-find-cmd.c:36) |
==20124== by 0x70CEA4: _prime (mongoc-cursor-find.c:61) |
==20124== by 0x70A73E: _call_transition (mongoc-cursor.c:1204) |
==20124== by 0x70A962: mongoc_cursor_next (mongoc-cursor.c:1280) |
==20124== by 0x6528F0: mongocxx::v_noabi::cursor::iterator::operator++() (cursor.cpp:45) |
==20124== by 0x652C02: mongocxx::v_noabi::cursor::iterator::iterator(mongocxx::v_noabi::cursor*) (cursor.cpp:80) |
==20124== by 0x652B2D: mongocxx::v_noabi::cursor::begin() (cursor.cpp:67) |
==20124== ... |
Valgrind command:
valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes --num-callers=200 ./executable |
Mongocxx version:
- custom build from c315129c7b70c304d894ea60b7df71d1f3a71acf (somewhere 3.6.3)
- related cmake flags:
- -DBUILD_SHARED_AND_STATIC_LIBS:BOOL=ON
- -DBUILD_SHARED_LIBS_WITH_STATIC_MONGOC:BOOL=ON
- -DMONGOCXX_ENABLE_SSL:BOOL=ON
- -DBSONCXX_POLY_USE_MNMLSTC:BOOL=ON
Mongoc version:
- custom build from 1.17.2
- related cmake flags:
- -DENABLE_AUTOMATIC_INIT_AND_CLEANUP:BOOL=OFF