[SERVER-81108] Sharded $search fails tassert in writeQueryStats Created: 15/Sep/23 Updated: 09/Nov/23 Resolved: 09/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.3.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Maddie Zechar | Assignee: | Maddie Zechar |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query Integration
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Sprint: | QI 2023-11-13 | ||||||||
| Participants: | |||||||||
| Description |
|
link to current test: https://github.com/10gen/mongo-enterprise-modules/blob/a5bf26ac4e16152bbab03ee1f24266042b866b93/jstests/search/sharded_search_query_stats.js
The above sharded $search test fails when queryStats is enabled (but passes when it is not enabled). Relevant snippet of backtrace is:
It appears killCursors are called, then writeQueryStats is called but keyGenerator is nullptr and thus fails the tassert. |
| Comments |
| Comment by Githook User [ 09/Nov/23 ] |
|
Author: {'name': 'madelinezec', 'email': 'mez2113@columbia.edu', 'username': 'madelinezec'}Message: |
| Comment by Maddie Zechar [ 01/Nov/23 ] |
|
When we enable queryStats on the shards of a sharded cluster, a cursor will be created for each shard. However, we make the queryStatsKey and put in behind a unique_ptr on CurOp. The ClientCursor constructor std::moves the queryStatsKey (which makes the key on CurOp then null) and copies over the queryStatsKeyHash as the latter is a cheap copy.
So in this way, the two shards are racing to call the ClientCursor constructor first, as the first one will own the key and have a copy of the hash while the second one will only have a copy of the hash. When mongos calls to dispose the cursors, it's only checked if the hash exists on the cursor before calling writeQueryStats. For the cursor that was constructed second, that check passes and it moves onto writeQueryStats() where it trips the tassert. |
| Comment by Charlie Swanson [ 01/Nov/23 ] |
|
Chatting with maddie.zechar@mongodb.com about this, we realized that this test is failing in a configuration we don't expect to support: the shards are configured to track query stats, but the mongos is not. It does not reproduce if only mongos is configured. I think this is still worth looking into and should be considered a bug, but I will move it to a later milestone in the project to reflect this lower priority. |