[SERVER-53953] Fix invariant failure when downgrading FCV with an index build in progress Created: 21/Jan/21  Updated: 29/Oct/23  Resolved: 23/Feb/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Bug Priority: Major - P3
Reporter: Ali Mir Assignee: Samyukta Lanka
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-52349 Enable feature flag for Remove the Ne... Closed
Related
related to SERVER-54703 Forced re-configs will restart the co... Closed
related to SERVER-54705 Test upgrade/downgrade with a concurr... Backlog
is related to SERVER-50423 Change memberConfig's slaveDelay fiel... Closed
is related to SERVER-54389 Audit internal uses of force reconfig... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Repl 2021-02-08, Repl 2021-02-22, Repl 2021-03-08
Participants:

 Description   

In the ticket SERVER-50423, we've added reconfigs to the upgrade/downgrade path that will change the name of the delay field name on replica set member configs. This behavior is hidden behind a feature flag. When the feature flag is enabled and we perform a reconfig on downgrade to change the field name from secondaryDelaySecs to slaveDelay, the test index_downgrade_fcv.js fails due to an invariant failure here. The error code is ReadConcernMajorityNotAvailableYet, which makes me suspect the index build component failed while attempting to establish a read concern majority cursor for the collection scan, as the system was in the middle of a reconfig. One potential solution is to busy-wait until read concern majority is available, which we seem to do here

As a note, we have disabled the index_downgrade_fcv.js test for feature flag enabled variants in SERVER-50423. We should make sure to enable this test again once this ticket is closed.



 Comments   
Comment by Githook User [ 23/Feb/21 ]

Author:

{'name': 'Samy Lanka', 'email': 'samy.lanka@mongodb.com', 'username': 'lankas'}

Message: SERVER-53953 Use safe reconfigs for setFCV command
Branch: master
https://github.com/mongodb/mongo/commit/d9f77934c145e2f602c3b5c72d7ce9ae6aec233a

Comment by Samyukta Lanka [ 18/Feb/21 ]

I think the root of the problem here is that forced reconfigs will bypass doing a noop write while also clearing the committed snapshot (as detailed in SERVER-54389).

For this specific case, using a safe reconfig instead should solve the problem since that will mean there will be a committed snapshot to read from. However, there is still the issue of force reconfigs triggering an invariant for index builds. I think that should be addressed in a separate ticket.

Generated at Thu Feb 08 05:32:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.