Add $minMaxScaler 'input.normalization' option to $scoreFusion stage

XMLWordPrintableJSON

    • Query Integration
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Once $minMaxScaler implementation complete, add "minMaxScaler" as an option to 'input.normalization'.

      To implement this, optionally add a $setWindowFields stage after the $addFields stage that computes the un-normalized score.

      So the score adding stage(s) will go from:

      {$addFields:
  {<input_pipeline_name>_score:
    {$multiply:
      [{"$score"}, <pipeline_weight>]
    },
  }
}

      to: 

      {$addFields:
  {<input_pipeline_name>_score:
    {$multiply:
      {"$score"}, <pipeline_weight>]
    },
  }
},
{$setWindowFields:
  {
    sortBy: { 
      <input_pipeline_name>_score: -1 // high scores first 
    }
    output: {
      <input_pipeline_name>_score: { 
        $minMaxScaler: {input: "$<input_pipeline_name>_score"},
    }
  }
}

      I haven't actually checked if its possible to make the input and output fields the same (effectively update / overwrite a field) with the $setWindowFields stage. However, if we get an error doing this, we should be able to call the unnormalized score (output of $addFields) one field, and then the normalized score (output of $setWindowFields) the final score field - and then after add a $project after that removes the unnormalized score.

            Assignee:
            Finley Lau
            Reporter:
            Adithi Raghavan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: