[SERVER-76087] implement $queryStats for $search and $searchMeta queries Created: 13/Apr/23  Updated: 29/Oct/23  Resolved: 15/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0

Type: Task Priority: Major - P3
Reporter: Davis Haupt (Inactive) Assignee: Maddie Zechar
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-85075 Tracking: Milestone 2 Closed
Related
related to SERVER-81124 Complete TODO listed in SERVER-76087 Closed
related to SERVER-82098 Complete TODO listed in SERVER-76087 Closed
Assigned Teams:
Query Optimization
Backwards Compatibility: Fully Compatible
Sprint: QO 2023-09-18
Participants:

 Description   

$search and $searchMeta queries should be serialized as a constant. This is already the case for $search, where 

 

    $search: {
        query: "cakes"
        path: "title"
    }
}

results in the respective key outputted from $queryStats:

 

    $search: "?obj"
}

This is not the case currently for $searchMeta and this ticket will implement correct serialization for $searchMeta for purposes of queryStats. 

 

This ticket should also address adding test coverage for $search and $searchMeta queries in an environment where $queryStats is enabled.



 Comments   
Comment by Maddie Zechar [ 13/Sep/23 ]

The design in the doc suggests shapifying 

{
  "$search": {
    "near": {
      "path": "released",
      "origin": ISODate("2011-09-01T00:00:00"),
      "pivot": 7776000000
    }
  }
}

to

{$search: "?obj"}

which I think is fine and seems to be the same as suggestion above and is also current behavior.

I think $searchMeta would ideally also be shapified to just `$searchMeta: "?obj"` for consistency across atlas search products. However, $searchMeta currently doesn't serialize the search query bson (the obj value for the field $searchMeta or $search) as a literal (whereas $search does). So a $searchMeta query like: 

{ 
    $searchMeta: {
        query: "cakes"
        path: "title"
    }
}

will have a queryStats key:

 { 
    $searchMeta: {
        query: "cakes"
        path: "title"
    }
}

If we are agreed that $searchMeta query should also be treated as a literal, then there is work to do in its serialize() method
 
 

Comment by Charlie Swanson [ 13/Sep/23 ]

maddie.zechar@mongodb.com is there engineering work to do here? What is the current behavior? When you say "redacted or made a constant" - did you have a particular thing in mind? Maybe just an empty object or {"?": "?"} (which I think is the representative object shape)?

So that would make 

{$search: { /* complex search grammar */ }}

turn into this:

// representative shape
{$search: {"?": "?"}}
// output from $queryStats (debug format)
{$search: "?object"}

 
How about $searchMeta - how will that show up?

Comment by Maddie Zechar [ 12/Sep/23 ]

Unlike regular find and agg queries, mongod does not parse the fields of a $search query into data structures that know the type of value their respective fields/operators hold. Rather, the raw bson of the $search query is preserved and passed to mongot. In this way the $search stage is a black box that only lucene inspects. Without this ability to glean meaning from the contents of the $search black box, it's not possible to serialize distinct $search operators/fields like we do for find and agg. Thus to support recording $queryStats metrics for $search queries, the $search stage should be redacted or made a constant.

Generated at Thu Feb 08 06:31:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.