Generalize generateCollScan's filter dependency analysis to clustered collscans

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      GenerateCollScan has a nifty optimization where it extends the set of fields passed to it by the top-level dependencies of the residual filter, saving a getField call.

      This, however, is not implemented for clustered collection scans.

      Repro:

      const d = db.getSiblingDB("clustered_filter_slot_minimal");
      d.dropDatabase();
      d.createCollection("clustered", {
        clusteredIndex: { key: { _id: 1 }, unique: true, name: "c_id" },
      });
      d.clustered.insertOne({});
      d.regular.insertOne({});
      
      // The bound makes this a clustered scan where possible
      // 'a' is a residual filter and is referenced nowhere else
      // $group forces the pipeline into SBE.
      const pipeline = (field) => [
        { $match: { [field]: { $gte: 0 }, a: 0 } },
        { $group: { _id: "$b" } },
      ];
      
      print("------ Clustered -----");
      
      // _id bound on a clustered collection -> clustered collection scan
      print(
        d.clustered.explain().aggregate(pipeline("_id")).queryPlanner.winningPlan
          .slotBasedPlan.stages,
      );
      
      print("------ Regular -----");
      
      // Force a collection scan by avoiding a predicate on the _id field
      print(
        d.regular.explain().aggregate(pipeline("c")).queryPlanner.winningPlan
          .slotBasedPlan.stages,
      );
      

      Output:

      ------ Clustered -----
      [3] project [s7 = newBsonObj("_id", s6)] 
      [3] group [s6] [] spillSlots[] mergingExprs[] 
      [3] project [s6 = (s5 ?: null)] 
      [1] filter {traverseF(getField(s3, "a"), lambda(l2.0) { ((move(l2.0) == 0) ?: false) }, false)} 
      [1] scan s1 = minRecordId, s2 = maxRecordId [s3 = record, s4 = recordId] [s5 = b] @"573407a8-9e56-4324-b619-45e263ca714a" forward 
      ------ Regular -----
      [3] project [s7 = newBsonObj("_id", s6)] 
      [3] group [s6] [] spillSlots[] mergingExprs[] 
      [3] project [s6 = (s3 ?: null)] 
      [1] filter {(traverseF(s4, lambda(l3.0) { ((move(l3.0) == 0) ?: false) }, false) && traverseF(s5, lambda(l4.0) { ((move(l4.0) >= 0) ?: false) }, false))} 
      [1] scan generic [s1 = record, s2 = recordId] [s3 = b, s4 = a, s5 = c] @"0ad84926-f9a9-4d95-a035-e9e0758e92a0" 
      

      Note that the regular collscan extracts s4 = a, whereas the clustered scan uses getField in the filter stage.

            Assignee:
            Unassigned
            Reporter:
            Kartal Kaan Bozdogan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: