355175510 12 Sep 2013 19:00:08 -0700 [Handler-15688] INFO c.n.m.m.service.AppUserServiceImpl - savePrefsIntoAppUser: query=

{ "userId" : "5210f35a0cf271f4bb861728"}

, update={ "$set" : { "prefs" :

{ "nfl_news" : true , "team_0325_gameStart" : true , "team_0325_finalscores" : true}

, "prefChangeDate" : { "$date" : "2013-09-13T02:00:08.244Z"}} , "$inc" : { "prefUpdateCount" : 1}}
355175510 12 Sep 2013 19:00:08 -0700 [Handler-15294] ERROR c.n.m.c.aop.RetryMethodInterceptor - =>> error = {}
com.nfl.mobile.commons.service.ServiceException:

{ "serverUsed" : "/127.0.0.1:27017" , "err" : "error inserting 1 documents to shard rep2:rep2/10.198.22.130:27017,10.198.22.132:27017,10.198.22.134:27017,10.198.22.136:27017,172.30.2.130:27017,172.30.2.132:27017 at version 11|37||51f4bff88d06f90d43f642a2 :: caused by :: socket exception [SEND_ERROR] for 10.198.22.132:27017" , "code" : 16460 , "n" : 0 , "ok" : 1.0}

at com.nfl.mobile.mongo.service.AppUserServiceImpl.register(AppUserServiceImpl.java:112) ~[mongodb-user-api-1.7.0.jar:na]
at com.nfl.mobile.mongo.service.AppUserServiceImpl$$EnhancerByGuice$$1dc893e6.CGLIB$register$0(<generated>) [sisu-guice-2.9.0.jar:na]

Checking Jira on mongo seems to indicate there could hostname issue.

https://jira.mongodb.org/browse/SERVER-10578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

I'm closing this as it was ultimately an issues with the hostnames of the servers. Even though all shards used unique hostnames in mongodb, it appears that the issue was that two systems had the same hostname at the OS level.