Similar to the etc/test_lifecycle.yml tag file from SERVER-28440, an etc/test_retrial.yml tag file should be added for marking tests with a "retry_on_failure*" tag. Since a human is intended to edit the etc/test_retrial.yml, there should be a header comment that explains how to be specific in indicating that

• the test sometimes fails in any task on any build variant with any distro,
• the test sometimes fails in a specific task on any build variant with any distro,
• the test sometimes fails in a specific task on a specific build variant with any distro, or
• the test sometimes fails in a specific task on a specific build variant with a specific distro.

by giving examples of the

• retry_on_failure,
• retry_on_failure|${task_name},
• retry_on_failure|${task_name}|${build_variant}, and
• retry_on_failure|${task_name}|${build_variant}|${distro_id} tags, respectively.

Additionally, there should be a warning that making it possible to ignore the test failures increases the risk we won't (or will take longer to) discover a bug and that therefore tests should be tagged judiciously.