I decided to run the test suite with verbose flags set. Because this would output verbose messages on the stdout, the python test would be marked as a failure. This prevents me from running the python test in a loop until it fails with a real error.

It will be useful to have a command line option to out python test suite, that would prevent the test from being marked a failure if stdout is not empty

We could consider creating a separate file for verbose output.