[SERVER-54357] Avoid normalizing partialFilterExpression when creating a new index Created: 05/Feb/21  Updated: 29/Oct/23  Resolved: 10/Mar/21

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Task Priority: Major - P3
Reporter: Bernard Gorman Assignee: Bernard Gorman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Sprint: Query Execution 2021-02-22, Query Execution 2021-03-08, Query Execution 2021-03-22
Participants:

 Description   

In SERVER-25023, we introduced improved normalization of index specs such that partialFilterExpression and wildcardProjection fields are converted to a canonical representation on index creation. This means that logically identical filters will appear the same in the catalog and in the output of listIndexes; before this ticket, only the collation field was normalized in this way.

However, this turns out to have caused some problems for the Cloud team. The Automation agent is responsible for taking an index configuration from the user and continuously ensuring that all nodes match these settings. They do so by issuing listIndexes and comparing the index specs against the desired set of indexes. Because the normalized partialFilterExpression can appear very different than the filter that the user specified, this means that Automation will need to be able to translate between the raw and normalized formats. This is not a trivial task, and will be fragile if we decide to allow additional expressions to be used in the partialFilterExpression or change how we optimize MatchExpression trees.

In practice, when comparing two indexes for equivalence, we do not actually rely on the serialized form; we parse the existing and candidate partialFilterExpressions and use MatchExpression::equivalent. It should therefore be safe to remove the normalization of partialFilterExpression, which means that Cloud can continue to expect the partialFilterExpression of existing indexes to exactly match the user's request. We should retain the normalization of wildcardProjection, though, since this will likely be leveraged by SERVER-47659 and is trivial to translate on Cloud's side.



 Comments   
Comment by Githook User [ 10/Mar/21 ]

Author:

{'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}

Message: SERVER-54357 Avoid normalizing partialFilterExpression when creating a new index
Branch: master
https://github.com/mongodb/mongo/commit/3f601b592620eb2abd207e4c57cff5bf51b4e07e

Generated at Thu Feb 08 05:33:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.