Uploaded image for project: 'C++ Driver'
  1. C++ Driver
  2. CXX-2306

mongocxx::cursor won't free cache memory

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Works as Designed
    • Icon: Critical - P2 Critical - P2
    • None
    • None
    • None
    • None

    Description

      Problem description:

      in our application, the incoming requests are queued and served by a thread from a thread pool.

      In this case, if we iterate over a result set with mongocxx::cursor, it seems, that cursor won't free up allocated memory and the process memory consumption will increase even that the cursor created locally and will be destroyed when it goes out of scope.

      (Note: we have developed a counter example, where every requests spawn a new thread and served on it (Please note: new thread, not the same thread with reuse), we couldn't observe such a huge memory consumption in this case)

      The sample code that performs a find and iteration:

      vector<Result> Query() {
        initializePoolOneTime();
       
        auto client =_pool->acquire();
        mongocxx::database db = (*client)["Database"];
        mongocxx::collection collection = db["keyword"];
        mongocxx::cursor cursor = collection.find(document{} << "XXX" << finalize);
       
        vector<Result> result;
        for(auto&& m : cursor) {
            Result c;
            c.chunk = getIntValue(m["YYY"]);
            chunks.push_back(c);
        }
      }
      

      Running our application by valgrind shows the following stack trace:

      ==20124== 524,288 bytes in 1 blocks are still reachable in loss record 1,065 of 1,065
      ==20124== at 0x4C33D2F: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
      ==20124== by 0x77C999: bson_realloc (bson-memory.c:154)
      ==20124== by 0x77CA04: bson_realloc_ctx (bson-memory.c:194)
      ==20124== by 0x75E162: _mongoc_buffer_fill (mongoc-buffer.c:254)
      ==20124== by 0x73324D: mongoc_stream_buffered_readv (mongoc-stream-buffered.c:240)
      ==20124== by 0x733D09: mongoc_stream_readv (mongoc-stream.c:237)
      ==20124== by 0x733E45: mongoc_stream_read (mongoc-stream.c:281)
      ==20124== by 0x75DF38: _mongoc_buffer_append_from_stream (mongoc-buffer.c:200)
      ==20124== by 0x6FB71C: mongoc_cluster_run_opmsg (mongoc-cluster.c:3468)
      ==20124== by 0x6F5A7B: mongoc_cluster_run_command_monitored (mongoc-cluster.c:544)
      ==20124== by 0x70A27A: _mongoc_cursor_run_command (mongoc-cursor.c:1052)
      ==20124== by 0x70BAA1: _mongoc_cursor_response_refresh (mongoc-cursor.c:1673)
      ==20124== by 0x70D09C: _prime (mongoc-cursor-find-cmd.c:36)
      ==20124== by 0x70CEA4: _prime (mongoc-cursor-find.c:61)
      ==20124== by 0x70A73E: _call_transition (mongoc-cursor.c:1204)
      ==20124== by 0x70A962: mongoc_cursor_next (mongoc-cursor.c:1280)
      ==20124== by 0x6528F0: mongocxx::v_noabi::cursor::iterator::operator++() (cursor.cpp:45)
      ==20124== by 0x652C02: mongocxx::v_noabi::cursor::iterator::iterator(mongocxx::v_noabi::cursor*) (cursor.cpp:80)
      ==20124== by 0x652B2D: mongocxx::v_noabi::cursor::begin() (cursor.cpp:67)
      ==20124== ...

      Valgrind command:

      valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --track-origins=yes --num-callers=200 ./executable

       

      Mongocxx version:

      • custom build from c315129c7b70c304d894ea60b7df71d1f3a71acf (somewhere 3.6.3)
      • related cmake flags:
        • -DBUILD_SHARED_AND_STATIC_LIBS:BOOL=ON
        • -DBUILD_SHARED_LIBS_WITH_STATIC_MONGOC:BOOL=ON
        • -DMONGOCXX_ENABLE_SSL:BOOL=ON
        • -DBSONCXX_POLY_USE_MNMLSTC:BOOL=ON

      Mongoc version:

      • custom build from 1.17.2
      • related cmake flags:
        • -DENABLE_AUTOMATIC_INIT_AND_CLEANUP:BOOL=OFF

      Attachments

        Activity

          People

            jesse.williamson@mongodb.com Jesse Williamson (Inactive)
            moravas8051@gmail.com norbert NNN
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: