It turns out that if you have an empty test_*.py file in your tree (empty in the sense of not containing at least one test class in it; can have other stuff), the test discovery code that figures out what to run if you don't ask for specific tests crashes on it.

The problem is that the sort pass assumes every file has at least one class in it. The fix is to discard empty files from the test collection before that point.