scoreDetails' field 'rawScore' is incorrect when $score references {$meta: 'score'}

XMLWordPrintableJSON

    • Query Integration
    • ALL
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None

      The desugaring of $score has 'scoreDetails' that reports the 'rawScore' (the direct result of the score expression argument) before normalization and weighting.

       

      Currently there is a problem when the score expression itself that references  the incoming {$meta: 'score'} because the score metadata has changed throughout the calculation, thus an incorrect 'rawScore' is presented to the user when we try to recompute the rawScore (directly using the current score metadata value).

      We should change the desugaring of $score to keep track of the 'rawScore' that is computed upfront, and reference that when setting 'scoreDetails' rather than re-doing the calculation.

       

      The new desugaring should look something like:

       

      {
        $replaceRoot: {
          “newRoot”: {
            “docs”: “$$ROOT”
          }
        }
      },
      {
        $addFields: {
          “rawScore”: <score_expression>
        }
      },
      {
        $setMetadata: {
          “score”: “$rawScore”
        }
      },
       
      <… optional stages for weights and normalization added here …>
       
      If scoreDetails is requested add stage:
      {
        $setMetadata: {
          “scoreDetails: {
            “value”: {
              “$meta”: “score”
            }
            … existing keys remain the same ..
            “rawScore”: “$rawScore”
          }
        }
      },
      {
        $replaceRoot: {
          “newRoot”: “$docs”
        }
      }
      

       

       

      So the desugaring logic will need to change in the following ways:

      • There will need to be an initial function that runs that adds the 3 initial stages, regardless of weights and normalization
      • There will need to a function at the end that runs at end that adds the final $replaceRoot stage
      • The $minMaxScaler normalization case will need to be modified to remove its $replaceRoot stages
      • The function / stage that adds the scoreDetails metadata will need to get the "rawScore" field from $rawScore

      I think the other logic should remain largely the same

       

      Make sure to add jstests where $score's score expression depends on an incoming score metadata. Add test pipelines that have multiple score stages. Of course, ensure that all the existing tests continue to pass

            Assignee:
            Finley Lau
            Reporter:
            Mariano Shaar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: