[DRIVERS-2690] Support sort by score $search stage Created: 02/Aug/23  Updated: 05/Jan/24

Status: Implementing
Project: Drivers
Component/s: None
Fix Version/s: None

Type: Spec Change Priority: Unknown
Reporter: Evan Darke Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: james+, jeff+
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Issue split
split to JAVA-5231 Support sort by score $search stage Backlog
split to PHPORM-105 Support sort by score $search stage Backlog
split to CSHARP-4832 Support sort by score $search stage Closed
Related
Epic Link: DRIVERS-2681
Driver Changes: Builder Changes Needed
Quarter: FY24Q4
Downstream Changes Summary:

Summary of necessary driver changes

  •  Read the DRIVERS ticket description for details

Commits for syncing spec/prose tests
(and/or refer to an existing language POC if needed)

  •  

Context for other referenced/linked tickets

  •  
Start date:
Driver Compliance:
Key Status/Resolution FixVersion
CSHARP-4832 Done 2.24.0
JAVA-5231 Backlog
PHPORM-105 Backlog

 Description   

Summary

Atlas Search has expanded syntax for the sort field options in $search. Previously, sort in $search only supported mapping field names to numbers (1 or -1). Driver support for this is already in progress (ticket). However, we already have a non-breaking change scheduled. Users will be able sort on "meta" fields, namely "searchScore", by providing a BsonDocument as a value. 

Minimal Examples

sort: { placeholder: {$meta: "searchScore", order: -1} } // Sort by score in descending order (equivalent to default search order)

sort: { placeholder: {$meta: "searchScore", order: 1} } // Sort by score in ascending order (inverse of default search order)

Where "placeholder" in the above example can be any unique key in the sort document (the presence of the $meta field will overwrite the key value) similar to the $sort aggregation stage (https://www.mongodb.com/docs/atlas/atlas-search/sort/#syntax).

Sort by score can be combined with tiebreaking on real fields. Example:

sort: { foo: 1, placeholder: {$meta: "searchScore", order: -1} }

sort: { placeholder: {$meta: "searchScore", order: 1}, foo: 1 }

A builders API should support constants for these two options. For example,

SortOptions.sortOptions.sort(SortField.SCORE_DEFAULT, new SortField("foo", ASC));

 

 

Dependencies

This enhancement is part of work to support sequential pagination, but it is not strictly required for all paginated use cases. It's also a useful standalone feature, and can be done before driver support for sequential pagination. In fact, the server will probably release support for sort by score before searchBefore/searchAfter.

 It is dependent on https://jira.mongodb.org/browse/DRIVERS-2681

 

Motivation

Who is the affected end user?

Any Java or C# users of the builders API

How does this affect the end user?

The builders API increases discoverability for features implemented in the $search stage. At best, users will be annoyed that they have to write sort options by in raw JSON to achieve sort by score. At worst, they won't know the feature exists in $search and instead will use the $sort aggregation stage which is ~10x slower.

 

Java and C# users already have a workaround available by writing searchOptions in raw JSON, but they very likely won't know this feature exists if they aren't routinely scouring the MongoDB website for documentation changes – they are probably accustomed to JavaDoc as the definitive documentation for supported features. 

How likely is it that this problem or use case will occur?

Sorting by score explicitly will likely be used by any user interested in implementing paginated queries sorted by relevance.

If the problem does occur, what are the consequences and how severe are they?

  1. If users aren't aware this feature is supported in $search, they may use $sort stage to achieve an equivalent result which is 10x slower.
  2. If users have to write JSON by hand, they are fairly likely to do so incorrectly. The default sort order for score is -1, which even throws off Atlas eng briefly. A constant for SortField.SCORE_REVERSE, SortField.SCORE_DEFAULT would probably may this much less error prone. 

Is this issue urgent?

This sort enhancement is approved, but not yet implemented. We expect to begin rolling this out to customers in 2-4 weeks.

Is this ticket required by a downstream team?

Needed by e.g. Atlas, Shell, Compass?

Is this ticket only for tests?

Improves API discoverability for users.

Acceptance Criteria

What specific requirements must be met to consider the design phase complete?


Generated at Thu Feb 08 08:26:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.