When a file is included or excluded explicitly by name (i.e. not by glob pattern or tag), then an error is raised if the test isn't found. This is done to ensure that the blacklist for a suite is updated when a test is renamed or deleted. We should consider improving the "ValueError: Unrecognized jstest" error message to be clearer about potentially needing to update an entry in the blacklist.
In master, deleting the test loglong.js causes a failure:
In this patch, I moved loglong.js to a new location and it causes the compile to fail: