Uploaded image for project: 'C Driver'
  1. C Driver
  2. CDRIVER-4262

Memory consumption possibly too high with cursor

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Works as Designed
    • Icon: Unknown Unknown
    • None
    • None
    • None
    • None

    Description

      Summary

      I was investigating a memory consumption behavior within my own application and I noticed that the Mongo C driver is consuming more memory than I thought was necessary. What I did is that I setup a fake DB with documents that are roughly 2MB in uncompressed size in BSON representation. I then created an application that uses a cursor to iterate over the documents with a batch size of two. I assumed that in any given moment, the memory consumption therefore lies around ~4MB. So I ran the application with Valgrind's Massif and noticed that the memory consumption was at peak, 8MB in size. So my guessing is that at some point the batch is copied instead of being moved.

      I've figured out that the driver copies the server reply in the function mongoc_cluster_run_opmsg from the statically allocated reply_local variable to the reply variable.  See this part here: https://github.com/mongodb/mongo-c-driver/blob/ded9ae5e9f2897a283305175aae8e1bbf4021c36/src/libmongoc/src/mongoc/mongoc-cluster.c#L3538. I'm quite unfamiliar with the C driver so I was not sure if it possible to "move" the memory to the final reply. 

      **I've attached the program and the output file of Valgrind.

      Environment

      Linux, Valgrind with driver version 1.20.1

      How to Reproduce

      • Populate the database using the script from the ZIP
      • Compile the example from the ZIP
      • Run Valgrind with valgrind --tool=massif ./build/build/application
      • Inspect the output file, e.g. using MassifVisualizer

      Attachments

        Activity

          People

            ezra.chung@mongodb.com Ezra Chung
            kai.takac@gmail.com Kai Takac
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: