[SERVER-46147] Update repair to fix multikey errors without performing an index rebuild Created: 14/Feb/20  Updated: 29/Oct/23  Resolved: 29/Jul/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.7.0, 4.4.10

Type: Improvement Priority: Major - P3
Reporter: Brian Lane Assignee: Fausto Leyva (Inactive)
Resolution: Fixed Votes: 0
Labels: intern_validate_improvements
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-49340 Add repair mode to validate for start... Closed
Related
related to SERVER-43074 Do not use a global variable to encod... Closed
related to SERVER-49937 Validate repair mode should fix up wi... Backlog
related to SERVER-46148 Update repair to fix multikey errors ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: Execution Team 2020-05-04, Execution Team 2020-07-27, Execution Team 2020-08-10
Participants:

 Description   

When we have validation errors with multikey flags such as:

"... is not multi-key but has more than one key in document"

would be great when running repair if we could actually correct these without needing to rebuild the index.

Proposed solution:

  • If an index is not multikey and a multikey document is found, then the index will be set to multikey and the multikey paths will be updated for that index.
  • If a multikey index's multikey paths do not cover a multikey document, then the index’s multikey paths will be updated.
  • If a multikey index has multikey paths that are not associated with any multikey document, this is not an error. This will be reported as a warning, and there will be no attempt to correct the index. To clear the warning, the index can be rebuilt by the user.


 Comments   
Comment by Githook User [ 17/Sep/21 ]

Author:

{'name': 'Faustoleyva54', 'email': 'fausto.leyva@mongodb.com', 'username': 'Faustoleyva54'}

Message: SERVER-46147 Update repair to fix multikey errors without performing an index rebuild

(cherry picked from commit 6dd3014)
Branch: v4.4
https://github.com/mongodb/mongo/commit/4ce9f655e2c88117015e256168f6891432df0ebf

Comment by Githook User [ 28/Jul/20 ]

Author:

{'name': 'Faustoleyva54', 'email': 'fausto.leyva@mongodb.com', 'username': 'Faustoleyva54'}

Message: SERVER-46147: Update repair to fix multikey errors without performing an index rebuild
Branch: master
https://github.com/mongodb/mongo/commit/6dd301449ec7a48e35a7114e93b888df9958217f

Comment by Fausto Leyva (Inactive) [ 27/Jul/20 ]

https://mongodbcr.appspot.com/629360001/

Comment by Louis Williams [ 01/Jul/20 ]

Could we make this validation repair behavior a standalone-only feature? That would be consistent with reIndex, which is also standalone-only.

Comment by Daniel Gottlieb (Inactive) [ 05/May/20 ]

An alternate implementation of ghost timestamping I mentioned to vamsi.krishna and chenhao.qu was asking WT to find an appropriate timestamp for an update. For our ghost timestamping cases, it would be min(<last update's durable_timestamp>, <stable_timestamp> + 1). We can also emulate this in MongoDB (with more stable_timestamp retrying for races) if WT exposed a document version's durable_timestamp value.

Comment by Eric Milkie [ 05/May/20 ]

I'm kind of torn. It would be really nice to have a way to correct these problems on a running system without needing to restart the process (twice!) to use --repair. But only the --repair solution is straightforward.

Comment by Louis Williams [ 05/May/20 ]

Just clarifying that there are no problems with a solution that only implements this behavior in --repair mode. The options that Dan suggested are to expand this behavior to work with the validate command on a live server.

Comment by Daniel Gottlieb (Inactive) [ 05/May/20 ]

Had a conversation with gregory.wlodarek and louis.williams. The trouble with correcting multikey values is that there's no natural optime associated with this write. We wanted to raise our known options with a greater audience. What we have include:

  • Use a ghost timestamp.
  • Don't use a timestamp – break rollback.
  • Write a no-op oplog entry – secondaries need a different solution.
  • Wait for all previous catalog updates to become majority committed, don't use a timestamp. Requires commit point for liveness.
  • Piggy-back the update with incoming writes to the catalog entry. Requires writes to that specific entry for liveness.
  • Make the updates in --repair.
Comment by Eric Milkie [ 06/Apr/20 ]

Another option is to add a special flag to the validate command to permit it to make data changes rather than just observe them.

Comment by Eric Milkie [ 27/Feb/20 ]

Repair doesn't run a validate-like scan ahead of time, so I'm not sure how it would identify this situation prior to rebuilding indexes. We would have to have the validate command save something durable for repair to look at, possibly? Sounds tricky.

Generated at Thu Feb 08 05:10:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.