Uploaded image for project: 'Mongoid'
  1. Mongoid
  2. MONGOID-5473

Nested eager loading is still not fixing N+1 Query Problems

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Gone away
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 7.0.13
    • Component/s: None
    • Labels:

      Nested Includes is not working 🚨 

      Looks like even using the nested includes approach, the N+1 Query Problem is still happening!

      I tried to use the nested includes included in this commit, requested by this feature request, but looks like it is not really avoiding the N+1 Query Problem for nested associations, and let me show why below:

      Scenario 🌍 

      class Sheet < ApplicationRecord
  include Mongoid::Document
  has_many :groups
  field :title, type: String
end
class Group < ApplicationRecord
  include Mongoid::Document
  belongs_to :sheet
  has_many :columns
end
class Column < ApplicationRecord
  include Mongoid::Document
  belongs_to :group
end

      Steps to Reproduce 🚶 ...

       

      Column.all.includes(group: [:sheet]).map do |column|
  column.group&.sheet&.title
end

      ⤷ Result: Still fetches N sheets as shown below

       

       

      MONGODB | mongo2:30002 req:4884 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"columns", "filter"=>{}, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00007f9eec2d0838 @seconds=1661976640, @increment=1>, "signature"=>{"hash"=><BSON::B...
MONGODB | mongo2:30002 req:4884 | pocket_api_development.find | SUCCEEDED | 0.012s
MONGODB | mongo2:30002 req:4885 conn:1:1 sconn:31 | pocket_api_development.getMore | STARTED | {"getMore"=>#<BSON::Int64:0x00007f9ed0a28340 @value=7552879692520447446>, "collection"=>"columns", "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00007f9eec2d0838 @seconds=1661976640, @increment=1>, "signature"=>{...
MONGODB | mongo2:30002 req:4885 | pocket_api_development.getMore | SUCCEEDED | 0.022s
MONGODB | mongo2:30002 req:4886 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"groups", "filter"=>{"_id"=>{"$in"=>[BSON::ObjectId('5f10f17a1ada720136adb66c'), BSON::ObjectId('5f11c9b31ada720136adb694'), BSON::ObjectId('5f00e4901ada7201415fffb3'), BSON::ObjectId('5f41df3c1ada72016f436b8c'), BSON::ObjectId('5f4bab791ada7...
MONGODB | mongo2:30002 req:4886 | pocket_api_development.find | SUCCEEDED | 0.018s
MONGODB | mongo2:30002 req:4887 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>{"$in"=>[BSON::ObjectId('5f10f07f1ada720136adb669'), BSON::ObjectId('5f11bbc41ada720136adb680'), BSON::ObjectId('5f00b36f1ada7201415fffb2'), BSON::ObjectId('5f41de9d1ada72016f436b8b'), BSON::ObjectId('5f4bab2e1ada7...
MONGODB | mongo2:30002 req:4887 | pocket_api_development.find | SUCCEEDED | 0.011s
MONGODB | mongo2:30002 req:4888 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f10f07f1ada720136adb669')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4888 | pocket_api_development.find | SUCCEEDED | 0.021s
MONGODB | mongo2:30002 req:4889 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f11bbc41ada720136adb680')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4889 | pocket_api_development.find | SUCCEEDED | 0.029s
MONGODB | mongo2:30002 req:4890 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f00b36f1ada7201415fffb2')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4890 | pocket_api_development.find | SUCCEEDED | 0.009s
MONGODB | mongo2:30002 req:4891 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f41de9d1ada72016f436b8b')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4891 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4892 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f4bab2e1ada720171c53ad5')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4892 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4893 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f4bb4201ada720171c53c49')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4893 | pocket_api_development.find | SUCCEEDED | 0.003s
MONGODB | mongo2:30002 req:4894 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f4c21941ada720171c54055')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4894 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4895 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f4c21941ada720171c54055')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4895 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4896 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f3d77b01ada720171f580b5')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4896 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4897 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f5c27ee1ada7201711f08b8')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4897 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4898 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f5cd09b1ada7201711f0b0e')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4898 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4899 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f5cdfc51ada7201711f0b46')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4899 | pocket_api_development.find | SUCCEEDED | 0.004s
MONGODB | mongo2:30002 req:4900 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f5b21fb1ada720171c540e3')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4900 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4901 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f6261cd1ada720171b1ebbc')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4901 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4902 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f6420c21ada720171b1ec8f')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4902 | pocket_api_development.find | SUCCEEDED | 0.003s
MONGODB | mongo2:30002 req:4903 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f66aadf1ada720171b1f0ed')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4903 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4904 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f6e08fb1ada72017528a517')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
MONGODB | mongo2:30002 req:4904 | pocket_api_development.find | SUCCEEDED | 0.002s
MONGODB | mongo2:30002 req:4905 conn:1:1 sconn:31 | pocket_api_development.find | STARTED | {"find"=>"sheets", "filter"=>{"_id"=>BSON::ObjectId('5f6e9f601ada720171d0baa4')}, "limit"=>1, "singleBatch"=>true, "$readPreference"=>{"mode"=>"primaryPreferred"}, "$db"=>"pocket_api_development", "$clusterTime"=>{"clusterTime"=>#<BSON::Timestamp:0x00...
[... ad infinitum ...]

      Conclusion 💬

      Looks we are still having N+1 Query Problem for nested associations, even after using the nested includes approach implemented on this changelog.

      Could you check why it is not working at all?

      Thanks!

      Versions 📍

      1. Ruby: ruby 2.6.8p205
      2. Rails: Rails 6.0.5.1
      4. Mongoid: 7.0.13
      5. Mongoid Includes: 3.0.0

            Assignee:
            neil.shweky@mongodb.com Neil Shweky (Inactive)
            Reporter:
            victorcorcos@gmail.com Victor Costa
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: