Exorbitant cost calculated for INDEXED_NESTED_LOOP_JOIN_EMBEDDING, inferior plan selected

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The cost of this rejected, but actually better, plan is exorbitant:

          {
            isCached: false,
            usedJoinOptimization: true,
            queryPlan: {
              stage: 'INDEXED_NESTED_LOOP_JOIN_EMBEDDING',
              planNodeId: 4,
              costEstimate: 591.6620058593751,
              cardinalityEstimate: 5840,
              estimatesMetadata: { ceSource: 'Metadata' },
              leftEmbeddingField: 'none',
              rightEmbeddingField: 'partsupp',
              joinPredicates: [ 'p_partkey = ps_partkey' ],
              inputStages: [
                {
                  stage: 'COLLSCAN',
                  planNodeId: 1,
                  costEstimate: 17.733469482421874,
                  cardinalityEstimate: 1460,
                  estimatesMetadata: { ceSource: 'Code' },
                  filter: { p_name: [Object] },
                  nss: 'plan_stability_join_opt.part',
                  direction: 'forward'
                },
                {
                  stage: 'FETCH',
                  planNodeId: 3,
                  nss: 'plan_stability_join_opt.partsupp',
                  inputStage: {
                    stage: 'INDEX_PROBE_NODE',
                    planNodeId: 2,
                    nss: 'plan_stability_join_opt.partsupp',
                    keyPattern: [Object],
                    indexName: 'ps_partkey_1',
                    isMultiKey: false,
                    isUnique: false,
                    isSparse: false,
                    isPartial: false,
                    indexVersion: 2
                  }
                }
              ]
            },
       
      
      

      The cost to perform 5840 index scans is calculated at ~591, which is way too much. This plan can never win against hash joins.

      To reproduce:

       const pipeline = [
       {"$lookup":{"from":"partsupp","localField":"p_partkey","foreignField":"ps_partkey","as":"partsupp"}},
       {"$unwind":"$partsupp"},
       {"$match":{"$and":[{"p_name":{"$regex": /^l/}}]}}
       ]db.adminCommand({setParameter: 1, internalEnableJoinOptimization: true});db.adminCommand({setParameter: 1, internalJoinReorderMode: "random"});
      db.adminCommand({setParameter: 1, internalRandomJoinOrderSeed: 2});
      db.part.aggregate(pipeline).explain('executionStats').executionStats.totalKeysExamined + db.part.aggregate(pipeline).explain('executionStats').executionStats.totalDocsExamined;
      db.part.aggregate(pipeline).explain('executionStats').executionStats.executionTimeMillis;
      db.part.aggregate(pipeline).explain().queryPlanner.winningPlan.queryPlan;
      db.adminCommand({setParameter: 1, internalJoinReorderMode: "bottomUp"});
      db.part.aggregate(pipeline).explain('executionStats').executionStats.totalKeysExamined + db.part.aggregate(pipeline).explain('executionStats').executionStats.totalDocsExamined;
      db.part.aggregate(pipeline).explain('executionStats').executionStats.executionTimeMillis;
      db.part.aggregate(pipeline).explain().queryPlanner.winningPlan.queryPlan; 

            Assignee:
            Unassigned
            Reporter:
            Philip Stoev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: