[SERVER-66331] Additional $searchMeta test coverage Created: 10/May/22  Updated: 29/Oct/23  Resolved: 15/Sep/22

Status: Closed
Project: Core Server
Component/s: Distributed Query Execution, Sharding
Affects Version/s: None
Fix Version/s: 6.0.2

Type: Bug Priority: Major - P3
Reporter: Evan Plotkin Assignee: Charlie Swanson
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File unsharded_logs.txt    
Issue Links:
Backports
Related
related to SERVER-65692 Avoid unnecessary cursors for $search... Closed
is related to SERVER-66037 Use $$SEARCH_META in $unionWith tests... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0
Steps To Reproduce:

These can be repro'd locally with mongodb 6.0.0-rc4

Sharded Collection With Docs On One Shard

1) Connect to mongos

2) Setup Collection

sh.enableSharding("myDb")
use myDb
sh.shardCollection("myDb.myCol", {"_id":1}) 
db.myCol.insertMany([
  {
    "_id": 0,
    "color": "red"
  },
  {
    "_id": 1,
    "color": "blue"
  },
  {
    "_id": 2,
    "color": "green"
  },
  {
    "_id": 3,
    "color": "red"
  },
  {
    "_id": 4,
    "color": "blue"
  },
  {
    "_id": 5,
    "color": "green"
  }
])

3) Define Index on the collection with the given mappings on "myDb.myCol"

{
    "mappings": {
    "dynamic": false,
    "fields": {
      "color": [
        {
          "type": "stringFacet"
        },
        {
          "type": "string"
        }
      ]
    }
  }
}

4) See that this aggregation works, using $search and $project{}

db.myCol.aggregate([
  {
    "$search": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  },
  {
    "$project": {
      "newField": "$$SEARCH_META"
    }
  }
]) 

5) See that this aggregation fails with the error: Must own one of the two arguments if there are documents in the batch{}

 

db.myCol.aggregate([
  {
    "$searchMeta": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  }
]) 

 

Unsharded Collection

1) Connect to mongos

2) Setup Collection

sh.enableSharding("myDb")
use myDb
db.myColUnsharded.insertMany([
  {
    "_id": 0,
    "color": "red"
  },
  {
    "_id": 1,
    "color": "blue"
  },
  {
    "_id": 2,
    "color": "green"
  },
  {
    "_id": 3,
    "color": "red"
  },
  {
    "_id": 4,
    "color": "blue"
  },
  {
    "_id": 5,
    "color": "green"
  }
])

3) Define Index on the collection with the given mappings on "myDb.myColUnsharded"

{
    "mappings": {
    "dynamic": false,
    "fields": {
      "color": [
        {
          "type": "stringFacet"
        },
        {
          "type": "string"
        }
      ]
    }
  }
}

4) See that this aggregation works, using $search and $project{}

db.myColUnsharded.aggregate([
  {
    "$search": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  },
  {
    "$project": {
      "newField": "$$SEARCH_META"
    }
  }
]) 

5) See that this aggregation fails and the mongod crashes.{}

db.myColUnsharded.aggregate([
  {
    "$searchMeta": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  }
]) 

 

 

Sprint: QO 2022-05-16, QO 2022-05-30, QO 2022-06-13, QO 2022-06-27, QO 2022-07-11, QO 2022-07-25, QO 2022-08-08
Participants:

 Description   

When testing with 6.0.0-rc4 in cloud-dev, I ran into a few issues with $searchMeta when run against an unsharded collection & against a sharded collection with all of its docs on a single shard.

When I run a $searchMeta query against a sharded collection, where docs are only present on a single shard, I receive this error:

 

{"t":{"$date":"2022-05-09T23:20:06.685+00:00"},"s":"I""c":"QUERY",    "id":4625501, "ctx":"conn27","msg":"Unable to establish remote cursors","attr":{"error":{"code":6253102,"codeName":"Location6253102","errmsg":"Must own one of the two arguments if there are documents in the batch"},"nRemotes":0}} 

When I run a $searchMeta query against a totally unsharded collection, the mongod running the query crashes.  I have attached this log of the mongod that crashed: unsharded_logs.txt(Within this log, I also ran the same $search query and accessed $$SEARCH_META successfully through $project)

 

In both scenarios, when I use a normal $search query with $project to access $$SEARCH_META, it works correctly.

I've also attached local repros.  Cloud-dev is seeing issues with the mongotHost parameter so can't repro there yet.



 Comments   
Comment by Githook User [ 15/Sep/22 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-66331 Additional test coverage
Branch: v6.0
https://github.com/10gen/mongo-enterprise-modules/commit/6d682293ad306afbdc6588fce2484a3394253a21

Comment by Charlie Swanson [ 15/Sep/22 ]

Re-opening to commit on the 6.0 branch as described in the linked BACKPORT.

Comment by Steve Tarzia [ 16/May/22 ]

There is additional search test coverage in the works in SERVER-66037, and it looks related to this ticket.

Comment by Evan Plotkin [ 10/May/22 ]

The mongotHost issue has been fixed and this can now be repro'd against cloud-dev.

This is my cluster

(To repro in your own cluster, follow the repro steps, using the UI search index creator to create your search indexes with the given mappings)

evan.basicString (Sharded collection with zone ranges splitting docs across each shard. $$SEARCH_META and $searchMeta both work)

evan.basicStringUnshared (Unsharded collection. $$SEARCH_META works. $searchMeta crashes the mongod)

evan.basicStringNoZones (Unsharded collection. $$SEARCH_META works. $searchMeta gives this error: `Must own one of the two arguments if there are documents in the batch`)

The queries I am running on these are:

$$SEARCH_META

[
  {
    "$search": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  },
  {
    "$project": {
      "newField": "$$SEARCH_META"
    }
  }
]

$searchMeta

[
  {
    "$searchMeta": {
        "facet": {
          "operator": {
            "exists": {
              "path": "color"
            }
          },
          "facets": {
            "colorFacet": {
              "type": "string",
              "path": "color",
            }
          }
        }
      
    }
  }
]

Generated at Thu Feb 08 06:05:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.