There are some collmod modifications which, at a replica set level, are not idempotent. These are generally the index modifications which include in their command responses the original value of the field and the new value of the field. If the coordinator hits some retryable error (such as a stepdown) and retries the local collmod against each shard, the response it will return will be different than if it had not retried.

For example, modifications to the "hidden" field for an index if succeeding in the first attempt will look like

{ "hidden_old" : false, "hidden_new" : true, "ok" : 1 }

while after a retry, the response will just look like

{"ok" : 1 }