On a low-latency local network - gigabit ethernet/infiniband, the time taken to:

        response, res_time = self.__simple_command(sock_info,
                                                   'admin',
                                                   {'ismaster': 1})

appears to dominate any network latency.

We're running a mongos per application server, as per:
http://docs.mongodb.org/manual/administration/sharded-clusters/#sharding-mongos
(with 100 application servers).

Our connection string is: "localhost:port,backup1:port,backup2:port"

Setting secondary_acceptable_latency_ms to 0.0000001 doesn't help, as frequently the round trip to one of the backup mongoses really is less (because network latency doesn't dominate). The result is that load isn't directed through the local mongos, and is instead bounced through one of the backup mongoses. This then seems to lead to lots of random socket SEND_ERRORS depending on the load of the cluster.

If localhost is in the set of available mongoses I think it would be better to choose that rather than picking one of the mongoses randomly.