[SERVER-53386] Allow unittests to explicitly set feature flags for certain tests Created: 16/Dec/20  Updated: 29/Oct/23  Resolved: 06/Mar/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Task Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: Pierlauro Sciarelli
Resolution: Fixed Votes: 0
Labels: sharding-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Sharding 2021-02-22, Sharding 2021-03-08
Participants:

 Description   

We should have a RAII type that allows for enabling or disabling a feature flag during execution of a unit test, or part of a unit test.

While in general unit tests should not rely on environmental settings such as command line arguments or setParameter values, some of our C++ unit tests for high level sharding or replication behavior do in fact depend on subsystems involving such parameters. Using an RAII type allows specific unit tests to select required behavior or even iterate over multiple settings where appropriate, while still maintaining the property that the unit test itself does not depend on command line arguments or similar.



 Comments   
Comment by Pierlauro Sciarelli [ 06/Mar/21 ]

JFI - as suggested by Max during code review - the solution has been generalized so that any server parameter can now be changed during unit tests by using the RAIIServerParameterControllerForTest.

Comment by Githook User [ 06/Mar/21 ]

Author:

{'name': 'Pierlauro Sciarelli', 'email': 'pierlauro.sciarelli@mongodb.com', 'username': 'pierlauro'}

Message: SERVER-53386 Allow unittests to explicitly set feature flags for certain tests
Branch: master
https://github.com/mongodb/mongo/commit/0fe8780da088b25bdc17d6b3e74e9de7b607e1d1

Comment by Geert Bosch [ 05/Jan/21 ]

After speaking with Max and rereading the above comments, I believe there is consensus around the RAII approach, so I updated the description and title of this ticket to match. I'll move this ticket from investigating to ready for work.

Comment by Geert Bosch [ 05/Jan/21 ]

Unlike testing mongod, I would not generally expect unittests to enable or disable features, just as we don't allow selecting different storage engines and the like. While we have a fair number of knobs for mongod, it is not clear that it would be a good idea to copy that for unittests.

Comment by Pierlauro Sciarelli [ 17/Dec/20 ]

I like the RAII proposal, we could provide a simple scoped objects to be used just for unit tests. If the idea goes forward, the flags will be driven by each test case so we may consider disabling unit test tasks in the "disabled feature flags" variant to avoid confusion.

Comment by Max Hirschhorn [ 16/Dec/20 ]

Unit tests calling into feature-guarded code should be executed just if the relative feature flag(s) are enabled.

In addition, all features flags should be enabled for unit tests executed in the "disabled feature flags" variant.

The "disabled feature flags" build variant depends on running tests through resmoke.py with the --mongodSetParameter command line option to enable specific feature flags. I think adding a new command line option to unit test executables to enable specific feature flags (with the idea of the option being set automatically by resmoke.py) has some negative consequences. In particular:

  • ninja +basename_of_test_file syntax wouldn't set this new option. Maybe there's some way to forward extra options?
  • gdb --args ./build/install/bin/xx_test would now require engineers to remember to set the option themselves.

Given that our testing infrastructure's awareness of feature flags is equivalent to its awareness of server parameters, I'm hoping we'll go about the situation with unit tests and feature flagging a different way. I feel like unit tests which care about a specific feature flag can explicitly enable it within the test much like they do with changing server parameter values today. And this ticket could be repurposed to offer RAII types like we have with FailPointEnableBlock to avoid polluting global state between tests.

I'd be curious to know about any projects which would benefit from multiple feature flags being enabled (one for their project and one for another project) but didn't know about which other project it was in advance where explicitly enumerating the combinations wasn't an option.

Generated at Thu Feb 08 05:30:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.