Summary

Update the Unified Test Runner spec to clearly state that the schema version should be increased for test runner changes, even if the schema itself is not changed. In this case, the previous schema should be copied without changes (e.g. schema-1.7.json is copied to schema-1.8.json and tests that depend on changed behavior in 1.8 should require that schema version).

Bumping the schema version should not be necessary for new operations, unless those operations end up being documented in the spec itself (e.g. Special Test Operations, custom entity operations such as iterateUntilDocumentOrError).

Motivation

Who is the affected end user?

Drivers and spec test authors.

How likely is it that this problem or use case will occur?

Occasional.

If the problem does occur, what are the consequences and how severe are they?

Recurring discussions in PRs over whether or not the schema version should be bumped.

Is this issue urgent?

No.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

Yes.