[SERVER-60824] Support non-blocking sort() on cluster key Created: 19/Oct/21  Updated: 29/Oct/23  Resolved: 12/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.0-rc0

Type: Task Priority: Major - P3
Reporter: Haley Connelly Assignee: Joel Redman (Inactive)
Resolution: Fixed Votes: 0
Labels: PM-2311-M2
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-60121 Enable $hint to use the clusterKey of... Closed
Duplicate
is duplicated by SERVER-60836 Enable streaming sorts against the cl... Closed
Problem/Incident
causes SERVER-65541 Coverity analysis defect 122081: Unin... Closed
causes SERVER-73423 CLUSTERED_IXSCAN with sort generates ... Closed
Related
related to SERVER-65502 Complete TODO listed in SERVER-60824 Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.0
Sprint: QO 2022-03-07, QO 2022-03-21, Execution Team 2021-12-27, Execution Team 2022-01-10, Execution Team 2022-01-24, QO 2022-04-04, QO 2022-04-18
Participants:

 Description   

This should add support for reverse 'hint' as well.

For time-series collections, this feature will support non-blocking sorts for queries directly against the buckets collection. Further work will be needed to support more efficient sorting by time against the time-series view.



 Comments   
Comment by Githook User [ 30/Jan/23 ]

Author:

{'name': 'James Wahlin', 'email': 'james@mongodb.com', 'username': 'jameswahlin'}

Message: SERVER-65502 Address SERVER-60824 TODOs
Branch: master
https://github.com/mongodb/mongo/commit/4af684221317dade6ca8d0d47b67476949a8f830

Comment by Joel Redman (Inactive) [ 12/Apr/22 ]

In the case that an index exists with a leading field matching the clustered key, that index will be chosen even if a fetch is required. This is due to a difference in how work occurs during a collection scan. For a standard index, the first work returns a value. For a collection scan (clustered or otherwise) the first return is always a WAIT. This appears to have something to do with support for tailable scans, but I wasn't able to determine the details. In any case, because of this WAIT, the collection scan always begins with a deficit for the multiplanner.

Comment by Githook User [ 12/Apr/22 ]

Author:

{'name': 'Joel Redman', 'email': 'joel.redman@mongodb.com', 'username': 'joredman'}

Message: SERVER-60824 Permit non-blocking sorts on clustered collection scans
Branch: master
https://github.com/mongodb/mongo/commit/8e4d331e8c20eff3ed50e3d6bb7a1b4a58aa1c16

Comment by Githook User [ 15/Feb/22 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-63601 Disable test in sharding_clustered_collections.yml

reshard_collection_failover_shutdown_basic.js need to be disabled until SERVER-60824
Branch: master
https://github.com/mongodb/mongo/commit/1df6ba1b92ccbc77869314dd9182c708bcd80ad4

Comment by Louis Williams [ 02/Feb/22 ]

The work in SERVER-62616 left a TODO for this ticket here.

Comment by Louis Williams [ 21/Jan/22 ]

We want to avoid trying to special-case for time-series and make this apply to all clustered collections. However, if necessary, we may need to split this up into another ticket for time-series support for this feature.

Comment by Haley Connelly [ 26/Oct/21 ]

Decided to send this over to QO and see if perhaps it belongs in PM-2556 if we don't find it urgent right now?

Comment by Haley Connelly [ 22/Oct/21 ]

Assigning this to storage execution backlog until we determine if it is something worth borrowing someone from Query Optimization for

Comment by Haley Connelly [ 20/Oct/21 ]

Originally, the idea was to mimic $natural sort/hint behavior similar to how $natural interprets a sort({$natural: +-1}) as a hint instead. 

However, the CanonicalQuery part of the code doesn't have access to the QueryPlannerParams which can be used to obtain the cluster key or the Collection() which also can be used to obtain the cluster key. 

When trying to introduce changes to the query planner / query analysis, we decided that someone with more expertise in this part of the codebase is more fit to make these changes.

Generated at Thu Feb 08 05:50:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.