|
Randolph Tan
Does evergreen log something when it decides to exclude a test from running in a multiversion suite? I'm trying to figure out why a test is not being run in master when running in multiversion suite but gets in the multiversion suite of the older branch when backported
tausif.rahman
Is there an entry in this file? https://github.com/10gen/mongo/blob/b99588b40e4f5c44ed976438f58ce2249626204c/etc/backports_required_for_multiversion_tests.yml
maxh
(I feel like it'd help to have a link to a multiversion run where the test isn't being run for some commit and what the test expected to be included is)
Randolph Tan
master version, kind of old, https://spruce.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_all_feature_flags_required_display_sharding_multiversion_patch_af1ad78010f1396ad09f2f96e16705598f45a54b_63d04d1a57e85a4c62fbeb63_23_01_24_21_27_11/execution-tasks?execution=0&sorts=STATUS%3AASC
Randolph Tan
backport version
Randolph Tan
https://spruce.mongodb.com/task/mongodb_mongo_v6.2_enterprise_rhel_80_64_bit_dynamic_required_sharding_last_continuous_2_linux_enterprise_patch_4fa7a897ed4677516bd6ef1e2f9d4298ec8b35de_63d98231850e61080e21a5a5_23_01_31_21_06_14/tests?execution=0&sortBy=STATUS&sortDir=ASC
Randolph Tan
both have entries on backports_required file, but I had to update the one on v6.2 branch with the new server ticket number because for some reason it is skipped in master
New
maxh
I don't see https://spruce.mongodb.com/version/63d98231850e61080e21a5a5/changes?sorts=STATUS%3AASC%3BBASE_STATUS%3ADESC changing the ticket number for the prepare_transaction_then_migrate.js entries in backports_required_for_multiversion_tests.yml. backports_required_for_multiversion_tests.yml for the 6.2 branch only has an entry for last-lts and doesn't have an entry for last-continuous
Randolph Tan
I did not change it here. I used this example only so it would be obviuos that it is being executed in the older branch
maxh
The rule is that both branches need the same file+ticket entry in the same section for the test to get run (edited)
Randolph Tan
But if when I look in master, it looks green because it did not get run
Randolph Tan
ah
Randolph Tan
because the older branch had the wrong entries it will never get run in master?
Randolph Tan
unless I 'fix' the older branch?
Randolph Tan
I thought it was based on the ticket number and it checks the git log
maxh
there's no SERVER-68361 entry on the 6.2 branch in last-continuous section so it won't run in sharding_multiversion_last_continuous and there is no SERVER-68361 sentry on the 6.0 branch in last-lts so it won't get run in sharding_multiversion_last_lts
maxh
I thought it was based on the ticket number and it checks the git log
Less clever than that. It relies on the byproduct of git cherry-pick making the same edits across the branches to that file
Randolph Tan
Hm... I see. I failed to update the ticket number when doing the original fix (twice actually from different tickets) because it didn't appear as a failure (probably because it was already not being run in master at that time). And the ticket number in the yml file is based on commit made in v5.1. I wonder if there's a better way of doing this so it would catch errors like this
maxh
It sounds like something would need to reconcile the Jira and/or git state to detect when an entry is omitted. Maybe
@tausif.rahman
can think on this with the rest of the SDP team. Perhaps there could be some kind of lint step when making changes on older branches which checks to see if there's an entry for the backported ticket on the later branch which hasn't been added to the older branch's backports_required_for_multiversion_tests.yml
:this:
1
Randolph Tan
Ok, I was so confused, but I think I know what happened
Randolph Tan
The entry was deleted for some reason
Randolph Tan
https://github.com/10gen/mongo/commit/7fbabf32d91ba028e6952b411aa2e7bfc253c342
maxh
heh must be a bug in garbage collection :kirby-sunglasses:
maxh
maybe there's a test case to turn this into for https://github.com/mongodb/mongo/blob/f8dba817139f2ea4493b6657f81e78cd8761b7c6/buildscripts/backports_required_for_multiversion_tests_deduplicator.py
Randolph Tan
If I understand it correctly, the ticket for cleaning up 'outdated' multiversion entry should have also deleted the entry in master
Randolph Tan
but it only deleted the one in v6.2 branch causing the 'anomaly'
tausif.rahman
I think we can improve this experience as part of PM-3104. I’ve created SERVER-73566 to track this
|