[SERVER-64410] Investigate whether we need to sort on searchScore when storedSource is true in $search Created: 10/Mar/22  Updated: 29/Oct/23  Resolved: 28/Mar/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.4.14, 6.0.0-rc0, 5.3.1, 5.0.7

Type: Bug Priority: Critical - P2
Reporter: Ted Tuckman Assignee: Ted Tuckman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Minor Change
Operating System: ALL
Backport Requested:
v5.3, v5.0, v4.4
Sprint: QO 2022-04-04
Participants:

 Description   

In a sharded environment, we currently add a sort while merging as part of the idLookup stage. However, idLookup is omitted when the storedSource flag is set on the search query, so the results may be merged in a different order. If users expect this implied sort, this could be considered a bug.

It is also worth noting that this blocks running stages in parallel on the shards after the idLookup. If we add this to storedSource, those queries could take a performance hit.



 Comments   
Comment by Githook User [ 08/Apr/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: v4.4
https://github.com/mongodb/mongo/commit/de31fa9031724250da584e34ef70e5fefddb1094

Comment by Githook User [ 08/Apr/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: v4.4
https://github.com/10gen/mongo-enterprise-modules/commit/30c6b3d450168740ef59fa026ba2e1b86876fd01

Comment by Githook User [ 28/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: v5.0
https://github.com/mongodb/mongo/commit/b977129dc70eed766cbee7e412d901ee213acbda

Comment by Githook User [ 28/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/6b4263a286cd2ddc15515a9d04a888314b06650c

Comment by Githook User [ 27/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline

(cherry-picked from commit 3b8aa12e9d9b82e5f8419e53578bfc6bde20114d)
Branch: v5.3
https://github.com/mongodb/mongo/commit/1625b8f241a12cdca16b51b480f5c78ce1802646

Comment by Githook User [ 27/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline

(cherry-picked from commit 3c4418bf1f3338d95060a78422b7d6051337131d)
Branch: v5.3
https://github.com/10gen/mongo-enterprise-modules/commit/125708fe12964c79b7d5d48a7fe8840fc845dc01

Comment by Githook User [ 26/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: master
https://github.com/mongodb/mongo/commit/3b8aa12e9d9b82e5f8419e53578bfc6bde20114d

Comment by Githook User [ 26/Mar/22 ]

Author:

{'name': 'Ted Tuckman', 'email': 'ted.tuckman@mongodb.com', 'username': 'TedTuckman'}

Message: SERVER-64410 Add ability to defer merge sort in pipeline
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/3c4418bf1f3338d95060a78422b7d6051337131d

Comment by Doug Tarr [ 21/Mar/22 ]

I'd recommend this be a P2 as this makes storedSource not work correctly in a sharded environment.

Comment by Kevin Rosendahl [ 21/Mar/22 ]

We definitely still need to sort on searchScore with storedSource, that's the $search contract (results ordered by score descending). I don't think anything about storedSource changes that.

What we should investigate in the perf spike later is whether or not we can omit mongot returning the results in sorted order if and only if there is a stage later in the pipeline that would make that sort order not observable.

But for today, we should definitely add the merge sort back in when storedSource is true.

cc alexander.lukyanchikov

Comment by Marcus Eagan (Inactive) [ 21/Mar/22 ]

Should we put this in the perf spike?

Comment by Marcus Eagan (Inactive) [ 21/Mar/22 ]

evan.plotkin and kevin.rosendahl

Generated at Thu Feb 08 06:00:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.