[SERVER-39723] Change listIndexes command behavior to show in-progress index builds Created: 21/Feb/19  Updated: 29/Oct/23  Resolved: 27/Feb/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.4.20, 3.6.11, 4.0.7, 4.1.9

Type: Task Priority: Major - P3
Reporter: Eric Milkie Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-39719 Ensure that a node performing an init... Closed
Documented
is documented by DOCS-14704 [SERVER] Investigate changes in SERVE... Closed
Problem/Incident
is caused by SERVER-25175 listIndexes shouldn't include in-prog... Closed
Related
related to SERVER-78576 Add originalSpec field to NewIndexSpe... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0, v3.6, v3.4
Sprint: Storage NYC 2019-02-25, Storage NYC 2019-03-11
Participants:
Story Points: 1

 Description   

The change in SERVER-25175 intended to fix a problem with initial sync as it interacts with its sync source to procure a list of indexes for each collection to build. If a background index build is in progress on the sync source and the sync source is a primary node, the background index build may or may not complete successfully. (Some examples of how it could fail are: 1. the node steps down 2. the background index build is interrupted by a killOp command 3. the background index build discovers some collection data that violates a constraint on the index being built, if the index spec has constraints.) The initial syncing node will proceed to attempt to build this same index, which might complete successfully even if the index build on the primary does not. This could leave the initial syncing node with an index that no other node has.

The initial sync process uses the listIndexes command on the sync source to obtain lists of indexes per collection. The attempt to fix this problem changed the listIndexes command to not report in-progress index builds (this will only affect background index builds, as foreground index builds lock the collection for their entirety and thus can never be observed in-progress by the listIndexes command). The fix had the intended effect for initial sync sync sources that are in primary state, but it also unintentionally changed the behavior of initial sync sync sources that are in secondary state. For such sources, background index builds are destined to complete successfully (since they already have done so on a primary node), and therefore cannot fail. Such index builds also write their oplog entries prior to completion, and therefore the only indication that an initial syncing node has to build the index is the listIndexes command response. Hiding in-progress index builds from initial sync for sync sources that are in secondary state could result in missing indexes for the initial sync. This code change restores the old behavior where listIndexes shows in-progress index builds.



 Comments   
Comment by Githook User [ 27/Feb/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-39723 Change listIndexes command behavior to show in-progress index builds
Branch: master
https://github.com/mongodb/mongo/commit/135a2a0a2a5b5b95e20bb83d9626036f946ba735

Comment by Githook User [ 25/Feb/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-39723 Change listIndexes command behavior to show in-progress index builds
Branch: v4.0
https://github.com/mongodb/mongo/commit/f5e0120d8b8ac05baea642f423abb55a5a8b8f65

Comment by Githook User [ 21/Feb/19 ]

Author:

{'name': 'Eric Milkie', 'username': 'milkie', 'email': 'milkie@10gen.com'}

Message: SERVER-39723 Revert "SERVER-25175 listIndexes command only includes ready indexes"

This reverts commit 1c922de9cac97065fb018f5d049566ecba803433.
Branch: v3.6
https://github.com/mongodb/mongo/commit/2151d1d219bbaff918db0d43fcb812231c20d53a

Comment by Githook User [ 21/Feb/19 ]

Author:

{'name': 'Eric Milkie', 'email': 'milkie@10gen.com', 'username': 'milkie'}

Message: SERVER-39723 Revert "SERVER-25175 listIndexes command only includes ready indexes"

This reverts commit e013b606016ead1c82ed7c09cbe3c558d5aaa3b2.
Branch: v3.4
https://github.com/mongodb/mongo/commit/f69dc2d3637dd1fb8de8d11c2a03289a5c31fceb

Generated at Thu Feb 08 04:52:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.