Memory leak when using cursors

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 2.24.0
    • Affects Version/s: 2.18.0, 2.21.1
    • Component/s: Query
    • None
    • Ruby Drivers
    • Not Needed
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      This leak seems to have started since v2.18.0.

      The following script demonstrates the issue:

      require 'bundler/inline'gemfile(true) do
        source "https://rubygems.org"
        gem "mongo", ENV["MONGO_VERSION"] || "2.21.1"
        gem "memory_profiler"
        gem "minitest", require: "minitest/autorun"
      endclient = Mongo::Client.new('mongodb://127.0.0.1:27017/cursor_leak')
      collection = client[:test_docs]module CursorLeakPatch
        def schedule_kill_cursor(*args, **kwargs, &block)
          super
          @cursor_reaper.kill_cursors if ENV["PATCH"]
        end
      end
      Mongo::Cluster.prepend CursorLeakPatchdescribe "Cursor memory leak" do
        before do
          ENV["PATCH"] = nil
          collection.delete_many
          collection.insert_many([{a: 1}, {a: 2}, {a: 3}])
        end  it "has 3 records" do
          assert collection.count == 3
        end  it "does not leak when batch_size equals limit" do
          report = MemoryProfiler.report(trace: [Mongo::Client]) { collection.find(nil, batch_size: 2, limit: 2).each{} }
          assert report.retained_objects_by_class.empty?
        end  it "leaks when batch_size < limit and records count" do
          report = MemoryProfiler.report(trace: [Mongo::Client]) do
            collection.find(nil, batch_size: 2, limit: 3).each{}
            sleep Mongo::Cluster::PeriodicExecutor::FREQUENCY + 1
          end
          refute report.retained_objects_by_class.empty?
        end  it "does not leak if we immediately kill cursors" do
          ENV["PATCH"] = "yes"
          report = MemoryProfiler.report(trace: [Mongo::Client]) { collection.find(nil, batch_size: 2, limit: 3).each{} }
          assert report.retained_objects_by_class.empty?
        end
      end 

      When a cursor has more results than the returned values in the first run, it would leak memory. Even after the periodic executor kills the cursors, we'd still notice the memory was leaked.

            Assignee:
            Dmitry Rybakov
            Reporter:
            Rodrigo Rosenfeld Rosas
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: