ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: 9.1.1
    • Component/s: None
    • None
    • None
    • Ruby Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      *Summary:* `run_targeted_callbacks` redefines a method on every call (its `respond_to?` check never sees the protected method it defines); under YJIT this retains every document

      *Affects:* Mongoid 9.1.1 and master (the check dates to 2012; the block-based `define_method` to 4.0.0, commit f4560f181). Ruby 4.0.5, ActiveSupport 8.1.3.1.

        1. Description

      `Mongoid::Interceptable#run_targeted_callbacks` is meant to build a callback method once per class and reuse it:

      ```ruby
      def run_targeted_callbacks(place, kind)
      name = "run_#

      {place}

      __#

      {kind}

      __callbacks"
      unless respond_to?(name)
      chain = ActiveSupport::Callbacks::CallbackChain.new(name, {})

      1. ...
        self.class.send :define_method, name do
      2. ...
        end
        self.class.send :protected, name
        end
        send(name)
        end
        ```

      The method it defines is made `protected`, and `respond_to?` returns false for protected methods. So the guard is always true: every call builds a new `CallbackChain` and calls `define_method` again on the document's class.

      This path is reached by `run_before_callbacks` / `run_after_callbacks`, which Mongoid itself calls from:

      • embedded batch insert: `Batchable#pre_process_batch_insert` / `#post_process_batch_insert` – e.g. assigning a whole list to an `embeds_many` on a persisted parent (`parent.items = [...]`), or `concat` of several documents
      • `HasMany::Proxy#save_or_delay` / `#persist_delayed` – `concat` of several new documents onto a persisted parent
      • `Clients::Sessions` commit/rollback callbacks
        1. Impact

      Without YJIT the cost is CPU: a method definition per call, and each one invalidates method caches for the class.

      With YJIT enabled (the Rails production default since 7.2), the replaced definitions are not freed. The block passed to `define_method` captures the binding of the call, so each retained definition keeps the document it was built for alive – and through it the parent and everything the parent loaded.

      In our production app (Sidekiq, Heroku, YJIT on) assigning embedded line items to a saved parent once per order kept ~3 million objects per process in a 30-minute burst of ~2,500 orders, until the process ran out of memory. Allocation tracing showed the kept `CallbackChain` objects (allocated at `interceptable.rb:404`) closely tracking the kept documents (e.g. 1,260 chains and 1,188 documents kept from one 15-minute window). Turning YJIT off for those processes stopped it.

        1. Reproduction

      No database needed: the documents are never saved.

      ```ruby
      require 'bundler/inline'

      gemfile do
      source 'https://rubygems.org'
      gem 'mongoid', '9.1.1'
      end

      require 'mongoid'

      class Item
      include Mongoid::Document
      field :payload, type: String

      1. count how often Mongoid (re)defines its targeted-callback methods on this class
        def self.method_added(name)
        $definitions += 1 if name.to_s.start_with?('run_')
        super
        end
        end
        $definitions = 0

      yjit = defined?(RubyVM::YJIT) && RubyVM::YJIT.enabled?
      puts "ruby=#

      {RUBY_VERSION}

      mongoid=#

      {Mongoid::VERSION}

      yjit=#

      {yjit.inspect}

      "

      name = 'runbeforesave_callbacks'
      Item.new.run_before_callbacks(:save)
      puts "after one call: protected_method_defined?=#

      {Item.protected_method_defined?(name)}

      respond_to?=#

      {Item.new.respond_to?(name)}

      "

      alive = -> { 3.times

      { GC.start }

      ; ObjectSpace.each_object(Item).count }
      start_alive = alive.()
      $definitions = 0
      10_000.times

      { Item.new(payload: 'x' * 1000).run_before_callbacks(:save, :create) }

      puts "10,000 calls: method definitions=#{$definitions} documents still alive=#

      {alive.() - start_alive}

      "
      ```

      Without YJIT (`ruby repro.rb`):

      ```
      ruby=4.0.5 mongoid=9.1.1 yjit=nil
      after one call: protected_method_defined?=true respond_to?=false
      10,000 calls: method definitions=20000 documents still alive=0
      ```

      20,000 definitions for 10,000 calls (one for `:save`, one for `:create`), where 2 are expected.

      With YJIT, the same loop run against one of our models in a production console (Ruby 4.0.5, Mongoid 9.1.1, `RubyVM::YJIT.enabled? == true`) kept every document:

      ```
      normal path (run_callbacks) rounds=2000 kept=0
      rebuild path (run_before_callbacks) rounds=2000 kept=1941
      rebuild path (run_before_callbacks) rounds=10000 kept=10000
      ```

      and YJIT's own memory (`RubyVM::YJIT.runtime_stats[:yjit_alloc_size]`) grew ~8 MB per 10,000 calls. The same run with YJIT off kept 0-1.

        1. Suggested fix

      Check for the method on the class itself, including protected methods and excluding ancestors (a subclass can have different callbacks than its parent):

      ```ruby
      unless self.class.method_defined?(name, false)
      ```

      `method_defined?` returns true for public and protected methods; passing `false` limits it to the class's own methods. One consequence: a callback added with `set_callback` after the first call would not reach the cached method – which is the intended design of the original code, but has not been the actual behavior since 2012. If that matters, an alternative is not to define a method at all and run the compiled chain directly on each call.

            Assignee:
            Unassigned
            Reporter:
            Dean Chen (EXT)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: