Our test extensions are signed using a test key, which we used to validate our test extensions at load time. This is only supported in non-secure builds, since secure builds don't allow providing a
extensionsSignaturePublicKeyPath parameter. Instead, secure builds use the mongot-extension production signing key to verify extensions being loaded into the server.
At the time of our initial design, release builds were not used in our CI pipelines. However, we now pass config=release to variants on the mongodb-mongo-master-nightly project. As a result, on the nightly runs, secure builds are used to run some of our tests, which fail to validate the test extension signatures given that they aren't signed with the secure mode key.
In previous instances of this issue, we explicitly disable featureFlagExtensionsApiSignatureValidation at the suite level (i.e vector_search.yml). In general, it's not easy to determine whether or not a test is being run against a secure mode build. The suites are shared between our nightly runs and our patch builds, which means turning off signature verification completely would reduce some of our test coverage.
To fix this issue in the nightly runs, turn off the feature flag in tasks that run on the nightly run. This will reduce some test coverage, but it is acceptable given that we have test coverage of signature verification in other places:
1. At the time of this writing, we've added a new set of variants that explicitly test secure mode for the mongot extensions.
2. We also have a unit test (load_extension_test.cpp), which performs the signature validation logic for our test extensions.
This means we have sufficient coverage for signature validation