Summary

We want CI builds to behave in accordance to the CI server name. Intermittent failures that we have take up team resources:

1. it's more difficult to understand whether we have an actual issue with the driver both when we release a driver and when we submit PRs;
2. we have to re-run Evergreen tasks, sometimes more than once, and that takes time.

Intermittent test failures can be classified as follows (this classification may not be exhaustive and may be expanded):

1. Not caused by bugs
   1. Caused by external resources.
   2. Caused by the test having known race conditions which are mitigated in practice by artificial delays being part of the test.
2. Caused by bugs
   1. Caused by the test having bugs, e.g., an unknown race condition.
   2. Caused by the server or driver bugs.

We should start with addressing failures that are not caused by bugs. Once they are addressed, we can assess the situations and decide what to do next.

Reports of some (ideally, all) intermittent and non-intermittent test failures can be seen here.

Motivation

Who is the affected end user?

The dbx-java team.

How does this affect the end user?

They are annoyed and less efficient than could have been. They also have increased chances of missing a bug.

How likely is it that this problem or use case will occur?

Intermittent failures happen all the time.

If the problem does occur, what are the consequences and how severe are they?

The consequences are tolerable at the moment, but we can do better.

Is this issue urgent?

No.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

Yes.

Cast of Characters

Not applicable.

Channels & Docs

Not applicable.