[SERVER-43666] Sort by "randVal" is not random Created: 26/Sep/19  Updated: 27/Oct/23  Resolved: 23/Oct/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Justin Seyster Assignee: David Storch
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-30405 add expression to generate a random n... Closed
Operating System: ALL
Steps To Reproduce:

MongoDB Enterprise > db.randval.insert([{a: 1}, {a: 2}, {a: 3}, {a: 4}, {a: 5}])
MongoDB Enterprise > db.randval.aggregate([{$sort: {rand: {$meta: "randVal"}}}])
{ "_id" : ObjectId("5d8d449cb417f08bc7302867"), "a" : 1 }
{ "_id" : ObjectId("5d8d449cb417f08bc7302868"), "a" : 2 }
{ "_id" : ObjectId("5d8d449cb417f08bc7302869"), "a" : 3 }
{ "_id" : ObjectId("5d8d449cb417f08bc730286a"), "a" : 4 }
{ "_id" : ObjectId("5d8d449cb417f08bc730286b"), "a" : 5 }

Sprint: Query 2019-11-04
Participants:

 Description   

The SortKeyGenerator::extractKeyPart() function gets a "randVal" from the Document metadata (by way of Expression::evaluate()) when the key pattern has {$meta: ""randVal"}. However, nowhere in the code base ever sets a random value for this piece of metadata, so the return value is always eoo. The sort deals with this lack of randomness by returning the documents in their natural order.

I discovered this problem when I attempted to add an invariant after this line to ensure that:

invariant(!plainKey.missing());

Currently, this invariant always fails, but when we figure out a way to ensure that documents have "randVal" metadata, we should be able to add that invariant in.



 Comments   
Comment by David Storch [ 23/Oct/19 ]

I'm closing this ticket as "Works as Designed". Sort by $meta:"randVal" was implemented internally in order to support $sample. It is not documented: you can see that it is never mentioned in the manual page for $sort.

If we ever do want to expose a query operator that causes the order of the result set to be randomized, we would need to either 1) fully implement and expose a "randVal" meta-sort, or 2) expect applications to rely on SERVER-30405, which would add an expression to generate a random number. In the latter case, users would need to do something like this:

db.collection.aggregate([..., {$addFields: {randVal: {$randVal: ...}}}, {$sort: {randVal: 1}}, ...]);

The behavior noted here was not introduced in 4.3 development, but rather exists in stable releases. I tested 4.2.0 in particular, though I suspect this behavior goes back to when $sample was first implemented.

Generated at Thu Feb 08 05:03:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.