When a driver gets a ReplicaSetMonitorRemoved error we want to be able to retry it since this exception is returned to the client. It's highly likely the operation would succeed on a different (or even the same) server. Ideally we would want a way to express this through a label, say RetryableError, but not tied directly to the type of operation (read/write) like RetryableWriteError does.