Current Situation

                             
WiredTiger compatibility tests are a set of tests running periodically on Evergreen to verify upgrade/downgrade compatibility across supported MongoDB release series and WT standalone versions.        

There are currently 9 compatibility test types across 3 build variants, including dirty restart, newer/older/patch releases, import compatibility, compatibility-against-develop (for PRs), WT standalone releases, and upgrade-to-latest tests.                                          

Pain Points 

                                                                                                                          
  1. Maintenance overhead — Two frameworks exist side-by-side:                                         
    - A monolithic shell script (compatibility_test_for_releases.sh) — grown large, hard to read, hard to run locally. 
    - A newer Python framework — better structure but underused and not kept up-to-date.                                             
  2. Slow local testing — You can pick the type of test but can't specify which release series to test against, so local runs are unnecessarily long.
  3. Minor release coverage — Minor releases (starting with 8.2) need the same compatibility test coverage as major releases, but this isn't fully in place yet.                                                                                                                                      
Improvement Suggestions   

                                                                      
  1. Deprecate the shell script — Migrate everything into the Python framework with a single config file for version management.
  2. Better local testing — Add a -v <release> flag to run tests against a specific release series only, rather than all of them.       
  3. Treat minor releases like major releases — Ensure all upgrade/downgrade paths from the Server Release Policy matrix are covered, including minor-to-major and minor-to-adjacent-minor paths.