Description

In normal operation, validations are not expected to time out. When validations are repeatedly skipped or do time out, it can be difficult to determine the underlying reason. Without enough diagnostic detail, it is hard to tell whether the behavior is expected, caused by the environment, or due to a product bug.

Mitigation

Logs

• Add individual shard success and failure logs for getting clone counts from donors and also getting deltas from donors and recipients
• Add duratins for the donor clone counts since those are expected to take a while and so having insights into how long those took would be valuable.

• Adding duration stats for _verifyClonedCollection like what we do for _verifyFinalCollection