[SERVER-47889] Hidden index tests should verify index filter behaviour Created: 01/May/20  Updated: 29/Oct/23  Resolved: 08/May/20

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: None
Fix Version/s: 4.4.0-rc6, 4.7.0

Type: Improvement Priority: Major - P3
Reporter: Bernard Gorman Assignee: Ruoxin Xu
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: Query 2020-05-18
Participants:

 Description   

In addition to their current coverage, the index filter tests should verify that:

  • When a query with a shape matching an index filter is executed, the index referenced by the filter is not used to answer the query if that index is currently hidden
  • If an alternative non-hidden index is available in the filter, it is used to answer the query. Otherwise, the planner outputs a COLLSCAN.
  • Un-hiding the index restores the index filter behaviour
  • It is legal to set an index filter on a hidden index, but the index will not actually be used until it is made visible


 Comments   
Comment by Githook User [ 08/May/20 ]

Author:

{'name': 'Ruoxin Xu', 'email': 'ruoxin.xu@mongodb.com', 'username': 'RuoxinXu'}

Message: SERVER-47889 Hidden index tests should verify index filter behaviour

(cherry picked from commit 3f124535f04261cb5b9c85528d0971e44ec77128)
Branch: v4.4
https://github.com/mongodb/mongo/commit/20ba463a85e3fd4af08e8047a7007865a12f8545

Comment by Githook User [ 08/May/20 ]

Author:

{'name': 'Ruoxin Xu', 'email': 'ruoxin.xu@mongodb.com', 'username': 'RuoxinXu'}

Message: SERVER-47889 Hidden index tests should verify index filter behaviour
Branch: master
https://github.com/mongodb/mongo/commit/3f124535f04261cb5b9c85528d0971e44ec77128

Comment by Bernard Gorman [ 07/May/20 ]

Re: ruoxin.xu's comment above:

We briefly considered changing the scope of this ticket to alter the behaviour of index filters as follows:

Consider the scenario where an index filter is set on index A, and there are 3 other indexes NOT in the filter that could answer the same query shape. If you then hide index A, the planner will NOT choose one of the other 3 indexes to answer the query; instead, it will produce a COLLSCAN. Hiding an index which is referenced in an index filter has the same effect as setting an index filter on a non-existent index; the planner will blindly apply the filters, match no indexes, and output a COLLSCAN, regardless of how many eligible indexes actually exist.

However, in the case where we have hidden all indexes captured by the filter, the more desirable behaviour would be to ignore the index filter altogether. Specifically:

  • If the index filter captures N indexes, and < N are hidden, then the filter is still applied because the remaining indexes are still eligible for filtering.
  • If all indexes captured by the filter are hidden, then the filter itself effectively becomes hidden. The filter is not applied at all and any non-filtered non-hidden indexes are eligible to answer the query.

However, we ultimately decided to leave the behaviour as-is. The purpose of hiding an index is to simulate what would happen if it were dropped, and we should therefore not add special logic to avoid the consequences just because an index filter is present. If we were to ignore the index filter when its associated index is hidden, so that the planner instead starts choosing non-filter indexes, then the user might be lulled into a false sense of security; dropping the hidden index with the index filter still present would immediately begin producing COLLSCANS instead, since the filter would now match zero indexes.

Comment by Ruoxin Xu [ 06/May/20 ]

Just to be clear about an edge case for the definition above - if the index filter captures 0 index, according to the definition, we should 'hide' this index filter as well. This may affect the current behaviour when the user sets a non-existent index, which would make the planner apply the filter filtering all the indexes and the user will just get a COLLSCAN. This change that enables 'hiding' a filter will cause the current behaviour for 'setting on a non-existent index' also 'hide' the filter. Do we also expect this behaviour for index filter?

Generated at Thu Feb 08 05:15:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.