Add Criteria#eager_load method to use aggregation pipeline for eager loading

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Fixed
    • Priority: Minor - P4
    • 9.1.0
    • Affects Version/s: None
    • Component/s: Associations
    • None
    • None
    • Fully Compatible
    • Needed
    • Hide

      1. What would you like to communicate to the user about this feature?

      This adds an `eager_load(...)` method to queries. The syntax is identical to the `includes` method, but it will use an aggregation pipeline to issue a single query, joining all the requested relations. This can result in significant performance gains in some cases, like with has_many or has_and_belongs_to_many associations.

      2. Would you like the user to see examples of the syntax and/or executable code and its output?

      The syntax is identical to `includes`; examples of `eager_load` can probably just reuse those examples.

      3. Which versions of the driver/connector does this apply to?

      This will be available in Mongoid 9.1 (when that is released).

      Show
      1. What would you like to communicate to the user about this feature? This adds an `eager_load(...)` method to queries. The syntax is identical to the `includes` method, but it will use an aggregation pipeline to issue a single query, joining all the requested relations. This can result in significant performance gains in some cases, like with has_many or has_and_belongs_to_many associations. 2. Would you like the user to see examples of the syntax and/or executable code and its output? The syntax is identical to `includes`; examples of `eager_load` can probably just reuse those examples. 3. Which versions of the driver/connector does this apply to? This will be available in Mongoid 9.1 (when that is released).
    • None
    • None
    • None
    • None
    • None
    • None

      Background

      ActiveRecord has 3 eager loading methods:

      • includes
      • preload
      • eager_load

      Of these:

      • AR's "preload" is exactly the same as Mongoid's "includes", it always does separate queries for each table.
      • AR's "eager_load" uses JOIN operators to return all the results in one query.
      • AR's "includes" behavior is generally the same as preload, except you can add a condition to the target object, in which case it uses a join like eager_load does.

      Proposal

      For Mongoid, I think we should add the "eager_load" method which would use the aggregation pipeline to do the same thing as a JOIN, i.e. fetch objects from multiple collections in one query. This may have better performance as it would not require to round-trip the database multiple times.

      My main concern is the performance for large result sets. I don't think the Agg Pipeline supports GETMORE cursors, and I'd have similar concerns as MONGOID-5730. Even with these limitations, .eager_load would still be useful in cases where the result set is expected to be smaller (e.g. using limit in the query.)

      This StackOverflow article is relevant:

      https://stackoverflow.com/questions/2350495/how-do-i-perform-the-sql-join-equivalent-in-mongodb

            Assignee:
            Jamis Buck
            Reporter:
            Johnny Shields
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: