minMaxScaler is slower when inside of subpipelines of $scoreFusion than in the top-level normalization field

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Integration
    • ALL
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Discovered during locust performance testing that this query:

      $scoreFusion: {
        input: {
          pipelines: {
            p1: {..., $score: {..., normalization: "minMaxScaler"}},
            p2: {..., $score: {..., normalization: "minMaxScaler"}}
          },
          normalization: "none"
        }
      }

      is much slower than:

      $scoreFusion: {
        input: {
          pipelines: {
            p1: {..., $score: {..., normalization: "none"}},
            p2: {..., $score: {..., normalization: "none"}}
          },
          normalization: "minMaxScaler"
        }
      } 

      Which should be computationally equivalent and desugar to a similar pipeline.

       

      We should understand why the first is so much less performant than the second, to both fix the issue, and understand if we have a flaw in our desugaring approach

              Assignee:
              Mariano Shaar
              Reporter:
              Mariano Shaar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: