[SERVER-80901] Audit gFeatureFlagServerlessChangeStreams Created: 08/Sep/23 Updated: 15/Nov/23 Resolved: 10/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.0.5 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Kyle Suarez | Assignee: | Romans Kasperovics |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Sprint: | QE 2023-10-02, QE 2023-10-16, QE 2023-10-30, QE 2023-11-13 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
This ticket has been split from an audit of all Query 7.0 feature flags. This ticket is a request to audit gFeatureFlagServerlessChangeStreams. Intial sync can temporarily reset the fcv value to uninitialized and sets the new value afterwards. This can cause call sites trying to inspect the fcv value to hit this invariant. We need to audit feature flag usage and determine if the feature flag check can be run during initial sync: If it can never be called when initial sync is running, do nothing. Note that this can be tricky to prove as we once thought the catalog cache loader can never be run while initial sync is happening but it can. If it might get run during initial sync, this could be the case if the feature is run during initial sync itself, if the feature is in a background thread that runs during initial sync, or if the feature is run in a command that is allowed during initial sync, such as hello, serverStatus, etc. In this case, use one of these options:
See this section of the README |
| Comments |
| Comment by Githook User [ 10/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Romans Kasperovics', 'email': 'romans.kasperovics@mongodb.com', 'username': 'romanskas'}Message: In mongod v7.0 'FeatureFlag::isEnabled()' crashes on invariant if Since 'featureFlagServerlessChangeStreams' is not set by default in v7.0, we | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Romans Kasperovics [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Attaching a test script to this ticket (because I am not going to commit it):
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Huayu Ouyang [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
romans.kasperovics@mongodb.com Yeah, I also see that gFeatureFlagServerlessChangeStreams isn't enabled by default on 7.0 anyways so it seems safe to replace isEnabled() with isEnabledUseLastLTSFCVWhenUninitialized on 7.0. And yes, this ticket is only for 7.0. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Romans Kasperovics [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks samy.lanka@mongodb.com! We (my team) think that replacing isEnabled() with isEnabledUseLastLTSFCVWhenUninitialized() is a valid possibility in 7.0, because it is unlikely that serverless change streams ever enabled for productive use in this version. I am not sure though that this is a safe choice apply for the same code in the subsequent versions. If this ticket is only about v7.0, I can replace isEnabled() with isEnabledUseLastLTSFCVWhenUninitialized(). Should I do that? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Samyukta Lanka [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
huayu.ouyang@mongodb.com I just wanted to point out that the audit of if the feature flag check is run during initial sync still needs to be completed and that checking isInitialized first isn't enough to avoid needing special logic for feature flag checks that can run during initial sync. romans.kasperovics@mongodb.com We are going to solve the race in a different way as part of | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Romans Kasperovics [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
samy.lanka@mongodb.com, this is a valid point. Do you think I should add a global lock acquisition to this code? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Huayu Ouyang [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
samy.lanka@mongodb.com I think that race is less specific to initial sync/this feature flag and we can fix it in | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Samyukta Lanka [ 06/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
romans.kasperovics@mongodb.com Isn't there still the possibility of a race condition if the FCV is reset after the call to serverGlobalParams.featureCompatibility.isVersionInitialized() but before the call to gFeatureFlagServerlessChangeStreams.isEnabled()? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Huayu Ouyang [ 02/Nov/23 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
romans.kasperovics@mongodb.com Yes, that's true, and also to clarify, this ticket ( |