[SERVER-64410] Investigate whether we need to sort on searchScore when storedSource is true in $search Created: 10/Mar/22 Updated: 29/Oct/23 Resolved: 28/Mar/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.14, 6.0.0-rc0, 5.3.1, 5.0.7 |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Ted Tuckman | Assignee: | Ted Tuckman |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Backwards Compatibility: | Minor Change | ||||
| Operating System: | ALL | ||||
| Backport Requested: |
v5.3, v5.0, v4.4
|
||||
| Sprint: | QO 2022-04-04 | ||||
| Participants: | |||||
| Description |
|
In a sharded environment, we currently add a sort while merging as part of the idLookup stage. However, idLookup is omitted when the storedSource flag is set on the search query, so the results may be merged in a different order. If users expect this implied sort, this could be considered a bug. It is also worth noting that this blocks running stages in parallel on the shards after the idLookup. If we add this to storedSource, those queries could take a performance hit. |
| Comments |
| Comment by Githook User [ 08/Apr/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Githook User [ 08/Apr/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Githook User [ 28/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Githook User [ 28/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Githook User [ 27/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: (cherry-picked from commit 3b8aa12e9d9b82e5f8419e53578bfc6bde20114d) |
| Comment by Githook User [ 27/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: (cherry-picked from commit 3c4418bf1f3338d95060a78422b7d6051337131d) |
| Comment by Githook User [ 26/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Githook User [ 26/Mar/22 ] |
|
Author: {'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}Message: |
| Comment by Doug Tarr [ 21/Mar/22 ] |
|
I'd recommend this be a P2 as this makes storedSource not work correctly in a sharded environment. |
| Comment by Kevin Rosendahl [ 21/Mar/22 ] |
|
We definitely still need to sort on searchScore with storedSource, that's the $search contract (results ordered by score descending). I don't think anything about storedSource changes that. What we should investigate in the perf spike later is whether or not we can omit mongot returning the results in sorted order if and only if there is a stage later in the pipeline that would make that sort order not observable. But for today, we should definitely add the merge sort back in when storedSource is true. |
| Comment by Marcus Eagan (Inactive) [ 21/Mar/22 ] |
|
Should we put this in the perf spike? |
| Comment by Marcus Eagan (Inactive) [ 21/Mar/22 ] |