[DOCS-14170] Investigate changes in SERVER-53675: Allow validate to fix up multikey metadata Created: 29/Jan/21  Updated: 13/Nov/23  Resolved: 19/Aug/21

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 4.9.0, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Ian Fogelman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-53675 Allow validate to fix up multikey met... Closed
Participants:
Days since reply: 2 years, 25 weeks ago
Epic Link: DOCSP-15042
Story Points: 2

 Description   

Description

Downstream Change Summary

Foreground validation on standalone nodes may fix up the following multikey metadata inconsistencies:

  • An index is multikey but there are no multikey fields
  • An index has multikeyPaths covering fields that are not multikey
  • An index does not have multikeyPaths but there are multikey documents (for indexes built before 3.4)

If any changes were made, a warning is included to the validate output and the 'repaired' flag is set to true.

Description of Linked Ticket

Index multikey metadata can either become inconsistent with collection data or more permissive than the data requires. There are three main ways this can happen:

  1. Indexes built before 3.4 (SERVER-15086) lack path-specific multikey metadata. All fields in an index are considered multikey and require blocking sorts. See SERVER-33387.
  2. Collection data has changed such that certain document paths are no longer multikey.
  3. Bugs: SERVER-48471SERVER-47694SERVER-43074, etc.

Multikey metadata may only be made less restrictive. If a collection's data has changed such that an index no longer has multikey documents or the multikey paths include more fields than are multikey, then the multikey metadata may be inconsistent. This is correct and allowed by the implementation, but it can lead to undesirable performance problems. The only solution is to reindex.

Collection validation scans all documents and generates multikey information, and it has the ability to fix index multikey metadata without returning an error. Allow foreground validation to "fix up" the following permitted inconsistencies:

  • An index is multikey but there are no multikey fields
  • An index has multikeyPaths covering fields that are not multikey
  • An index does not have multikeyPaths but there are multikey documents (for pre-3.4 indexes)

 

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 18/Aug/21 ]

Author:

{'name': 'ian fogelman', 'email': 'ian.fogelman@mongodb.com', 'username': 'ianf-mongodb'}

Message: DOCS-14170 Add foreground validation reference for 5.0 standalone mulikey index considerations
Branch: master
https://github.com/mongodb/docs/commit/57f7615c711d185f438606bb3e85cf42ef6a8179

Comment by Githook User [ 17/Aug/21 ]

EDIT: This message caused by having the wrong ticket number in an unrelated commit. This change has not yet been merged.

Comment by Louis Williams [ 04/Aug/21 ]

Yeah, all 3 of these changes are new and different from the currently documented behavior. It's up to you how much detail we want to include. The multikey state is pretty complex and might require some background. Aside from reporting whether an index is multikey, the multikeyPaths are not normally observable to users.

Comment by Ian Fogelman [ 04/Aug/21 ]

Hi louis.williams this seems to have been taken care of in a previous commit https://github.com/mongodb/docs/commit/3ab1290e4195b2e198c9436eb115ddde99beeed3 , was there anything additional that you would suggest? If not I will close this ticket.

Generated at Thu Feb 08 08:09:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.