See my comment on BF-18211 for details of the bug.
We should change line 339 to use utils.remove_if_exists() to not throw an exception if there is a benign race with after_test().