Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-85694

$searchMeta aggregation pipeline stage not passing correct query to mongot after PlanShardedSearch

XMLWordPrintableJSON

    • Query Execution
    • Fully Compatible
    • ALL
    • v7.3, v7.2, v7.0, v6.0
    • Hide

      On a sharded cluster running 7.2.0:

      Insert a document in collection "drug data":

      {
  id: '12345',
  openfda: {
    manufacturer_name: 'CSS Pharmacy',
    route: [
      'ORAL'
    ]
  }
}

      Create a search index named "drugs" over collection "drug_data":

      {
  "name": "drugs",
  "mappings": {
    "dynamic": false,
    "fields": {
      "id": {
        "analyzer": "lucene.keyword",
        "searchAnalyzer": "lucene.keyword",
        "type": "string"
      },
      "openfda": {
        "fields": {
          "manufacturer_name": [
            {
              "type": "string"
            },
            {
              "type": "token"
            },
            {
              "type": "stringFacet"
            }
          ],
          "route": [
            {
              "type": "string"
            },
            {
              "type": "stringFacet"
            }
          ]
        },
        "type": "document"
      }
    }
  }
}

      Run a query over collection "drug_data":

      db.drug_data.aggregate([
  {
    '$searchMeta': {
      index: 'drugs',
      facet: {
        operator: {
          exists: {
            path: 'id'
          }
        },
        facets: {
          manufacturers: {
            type: 'string',
            path: 'openfda.manufacturer_name',
            numBuckets: 10
          },
          routes: {
            type: 'string',
            path: 'openfda.route',
            numBuckets: 10
          }
        }
      }
    }
  }
])

      Observe error message:

      MongoServerError: mongot returned an error :: caused by :: Query should contain either operator or collector
      Show
      On a sharded cluster running 7.2.0: Insert a document in collection "drug data": {   id: '12345' ,   openfda: {     manufacturer_name: 'CSS Pharmacy' ,     route: [       'ORAL'     ]   } } Create a search index named "drugs" over collection "drug_data": { "name" : "drugs" , "mappings" : { "dynamic" : false , "fields" : { "id" : { "analyzer" : "lucene.keyword" , "searchAnalyzer" : "lucene.keyword" , "type" : "string" }, "openfda" : { "fields" : { "manufacturer_name" : [ { "type" : "string" }, { "type" : "token" }, { "type" : "stringFacet" } ], "route" : [ { "type" : "string" }, { "type" : "stringFacet" } ] }, "type" : "document" } } } } Run a query over collection "drug_data": db.drug_data.aggregate([ { '$searchMeta' : { index: 'drugs' , facet: { operator : { exists: { path: 'id' } }, facets: { manufacturers: { type: 'string' , path: 'openfda.manufacturer_name' , numBuckets: 10 }, routes: { type: 'string' , path: 'openfda.route' , numBuckets: 10 } } } } } ]) Observe error message: MongoServerError: mongot returned an error :: caused by :: Query should contain either operator or collector

      In sharded 7.2 clusters, the object inside $searchMeta is not being correctly passed to shards after PlanShardedSearch

      When running an aggregation pipeline like

      [
  {
    '$searchMeta': {
      index: 'drugs',
      facet: {
        operator: {
          exists: {
            path: 'id'
          }
        },
        facets: {
          manufacturers: {
            type: 'string',
            path: 'openfda.manufacturer_name',
            numBuckets: 10
          },
          routes: {
            type: 'string',
            path: 'openfda.route',
            numBuckets: 10
          }
        }
      }
    }
  }
]

      We can see this line in logs (verbosity set to 5):

      mongos_1   | {"t":{"$date":"2024-01-24T23:16:42.756+00:00"},"s":"D4", "c":"ASIO",     "id":22596,   "ctx":"conn63","msg":"startCommand","attr":{"request":"RemoteCommand 3610 -- target:[mongod3.internal:27017] db:ClinicalTrials cmd:{ aggregate: \"drug_data\", pipeline: [ { $searchMeta: { mongotQuery: { index: \"drugs\", facet: { operator: { exists: { path: \"id\" } }, facets: { manufacturers: { type: \"string\", path: \"openfda.manufacturer_name\", numBuckets: 10 }, routes: { type: \"string\", path: \"openfda.route\", numBuckets: 10 } } } }, metadataMergeProtocolVersion: 1, limit: 0, sortSpec: { $searchScore: -1 } } } ], cursor: { batchSize: 101 }, let: { NOW: { $literal: new Date(1706138202749) }, CLUSTER_TIME: { $literal: Timestamp(1706138199, 1) } }, fromMongos: true, readConcern: { level: \"local\", provenance: \"implicitDefault\" }, writeConcern: { w: \"majority\", wtimeout: 0, provenance: \"implicitDefault\" }, shardVersion: { e: ObjectId('000000000000000000000000'), t: Timestamp(0, 0), v: Timestamp(0, 0) }, databaseVersion: { uuid: UUID(\"d802bb99-99e8-459c-9cbe-f2f0022b87ba\"), timestamp: Timestamp(1706136857, 2), lastMod: 1 }, clientOperationKey: UUID(\"b007cb63-4766-4610-ad39-7581861db2b5\"), lsid: { id: UUID(\"b3530e27-f57f-47a4-8fc0-04a7cc551bed\"), uid: BinData(0, A009A5C38A39FA832F5D8E5FA067A58CC80A0E8F29A0F501773556AC0B1B33AD) } }"}}

      The relevant part of this line, post-formatting, is:

      {
  pipeline: [
    {
      $searchMeta: {
        mongotQuery: {
          index: "drugs",
          facet: {
            operator: {
              exists: {
                path: "id"
              }
            },
            facets: {
              manufacturers: {
                type: "string",
                path: "openfda.manufacturer_name",
                numBuckets: 10
              },
              routes: {
                type: "string",
                path: "openfda.route",
                numBuckets: 10
              }
            }
          }
        },
        metadataMergeProtocolVersion: 1,
        limit: 0,
        sortSpec: {
          $searchScore: -1
        }
      }
    }
  ]
}

      Notice that in this pipeline, there is a "mongotQuery" object inside $searchMeta - this object should not exist. Keys+values inside this mongotQuery object should be children of the $searchMeta object.

      This is making it so that users are unable to use search facets on sharded clusters on version 7.2.0.

            Assignee:
            charlie.swanson@mongodb.com Charlie Swanson
            Reporter:
            evan.nixon@mongodb.com Evan Nixon
            Votes:
            0 Vote for this issue
            Watchers:
            21 Start watching this issue

              Created:
              Updated:
              Resolved: