[SERVER-68685] Add new $_internalIndexKey agg expression Created: 09/Aug/22  Updated: 29/Oct/23  Resolved: 25/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.3.0-rc0

Type: New Feature Priority: Major - P3
Reporter: Randolph Tan Assignee: Rishab Joshi (Inactive)
Resolution: Fixed Votes: 0
Labels: query-director-triage, query-offsite
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-68687 Update Global Index fetcher pipeline Backlog
Related
Backwards Compatibility: Fully Compatible
Sprint: QE 2022-12-12, QE 2022-12-26, QE 2023-01-09
Participants:
Linked BF Score: 10

 Description   

Takes in a document, outputs an array of new objects that have their values converted into their index representation. Any fields in the input document not included in the "key" spec will be stripped in the output. If the index value contains a multikey index entry, then the output array will contain multiple documents, otherwise it will only contain one document. This expression should also be exempted from the 100MB limit.

 

Field description
doc input document
spec index key spec (format is similar to createIndex)


 Comments   
Comment by Githook User [ 25/Dec/22 ]

Author:

{'name': 'Rishab Joshi', 'email': 'rishab.joshi@mongodb.com', 'username': 'rishvin'}

Message: SERVER-68685 Adds new $_internalIndexKey agg expression.
Branch: master
https://github.com/mongodb/mongo/commit/6e7338e29df42e7119ecfc6916ee791c34cd27ac

Comment by Githook User [ 25/Dec/22 ]

Author:

{'name': 'Rishab Joshi', 'email': 'rishab.joshi@mongodb.com', 'username': 'rishvin'}

Message: SERVER-68685 Adds new $_internalIndexKey agg expression.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/e92ed23125d958a2f04cdbc9ed6a11394ca470ac

Comment by Arun Banala [ 10/Oct/22 ]

I see! So what you are looking for is more of a "generateIndexKey" expression, rather than reading an existing index key. That makes sense! Then you are right, it wouldn't fit into the $meta : "indexKey" expression.

Comment by Max Hirschhorn [ 10/Oct/22 ]

Thanks arun.banala@mongodb.com for the suggestion. I believe the reason we aren't pursuing a syntax around {$meta: "indexKey"} is mainly because the "indexKey" meta field is populated by the IndexScan stage for the currently executing query plan. The goal of the $_internalIndexKey expression is to use the BtreeKeyGenerator class directly to generate index keys for an explicitly specified index specification independent of the index used by the query plan. Changing the semantics of the {$meta: "indexKey"} seems—at least to me—like it'd be more invasive.

Comment by Arun Banala [ 09/Oct/22 ]

We already have a $meta: "indexKey" which does this, but it's not very helpful in the presence of multi-key indexes. The current implementation returns only one key per document and ignores the rest of keys. We can extend the syntax to support returning multiple documents for multi-key indexes. A new syntax like $meta: {type: "indexKey", options: {...}} would make it more extensible.

Comment by Kyle Suarez [ 28/Sep/22 ]

Flagging for scheduling; it is less urgent for Sharding than SERVER-68686 but randolph@mongodb.com would like it to be done within the next 2 months.

Generated at Thu Feb 08 06:11:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.