The mirrored reads code triggers a fatal assertion if the response from the mirroring request is non-ok: https://github.com/mongodb/mongo/blob/1373280c254a39d2ca7d85563718a6f74c927216/src/mongo/db/mirror_maestro.cpp#L382-L386

As far as I can tell, there's no basis for that assumption, the command could fail for any number of reasons, including network errors.