Context:

We currently have a small set of Evergreen tests that use of our CMake builds. Our CMake evergreen tests are limited to executing the CMake builds on our supported platforms/architectures and running the python & smoke test suites. The bulk of our Evergreen testing still uses the autoconf/libtool build e.g. format, PR tests, compiler tests, UB|ASan tests, csuite, compatibility tests etc.

We initially only added a subset of our tests with CMake support as to reduce testing overlap whilst CMake was new and being incrementally adopted. Given the increased adoption of CMake and the goal to eventually deprecate autoconf/libtool, we should start converting the bulk of our tests to use CMake builds. Note converting here means rewriting the autoconf tests to use CMake. Ideally we don't want to maintain duplicate sets of tests.

Definition of Done:

• Convert the majority of autoconf based tests to CMake based builds. A list of potential evergreen variants (including all tests under the given variant) could include:
  • Ubuntu 20.04 Variant
  • Ubuntu 20.04 ASAN Variant
  • Ubuntu 20.04 MSAN Variant
  • Ubuntu 20.04 UBSAN Variant
  • Ubuntu 20.04 Compilers
  • Ubuntu 20.04 Stress tests
  • RHEL 8.0 Variant
  • OS X 10.14 Variant
  • Windows 64-bit Variant
  • Cppsuite Stress Tests
  • Compatibility tests
  • Large scale tests

• Ensure CMake builds are used in our evergreen PR testing
• Delete current CMake evergreen variants - OS X 10.14 CMake, Ubuntu 20.04 CMake, Ubuntu 20.04 PPC CMake, Ubuntu 20.04 zSeries CMake, Windows 64-bit CMake
  • The tests in these CMake variants would be implemented/covered in the aforementioned autoconf variants. Hence making the CMake variants unnecessary once the other variants are converted.