[SERVER-74973] Track start and end datetimes of index builds and surface them in collection validation error results Created: 17/Mar/23  Updated: 16/Nov/23

Status: Backlog
Project: Core Server
Component/s: Catalog, Index Maintenance
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Backlog - Catalog and Routing
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-78968 Track collection created time and sta... Backlog
Assigned Teams:
Catalog and Routing
Sprint: Execution Team 2023-05-15, Execution Team 2023-05-29, Execution NAMR Team 2023-07-10, Execution NAMR Team 2023-08-07, Execution EMEA Team 2023-10-16, Execution EMEA Team 2023-10-30
Participants:
Story Points: 3

 Description   

Concurrent writes during index builds have historically been a source of index corruption bugs. Knowing when an index build started and ended its initial clone and when the index became ready may be helpful in determining whether a write to a document with bad index keys happened in the midst of the index build or not.



 Comments   
Comment by Haley Connelly [ 07/Nov/23 ]

After some discussion with josef.ahmad@mongodb.com, we think this ticket should aim to durably persist index start and end time on a separate, internal table with the sole purpose of observability and diagnosability. 

Whoever takes over this ticket will need to make some design decisions, such as what information we want in the collection (would it be useful to track additional metrics such as when the last collMod occurred, etc?) and when the information is safe to be removed. 

Comment by Haley Connelly [ 16/Oct/23 ]

For clarification, the plan is to record the wall time of when the index build starts and is finished for simplicity. 

cc louis.williams@mongodb.com 

Generated at Thu Feb 08 06:29:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.