lamont.nelson says: 
The one thing I can think of that may be problematic would be that the RSM publishes to ShardRegistry, and I think one other listeners in the code so we just have to make sure it doesn't publish to a dead object gracefully handles publishing if an object is being shutdown. The shutdown path for the servers is a bit messy and there are implicit (and explicit) dependencies in the ordering of things. 
 
 
also right now remove will set that flag (I think it's  an atomic _isDropped in the new version).
 
 
basically in the servers the rsm starts pretty early and ends pretty late due to the ShardRegistry holding on to a targeter pointer.
 
See https://jira.mongodb.org/browse/SERVER-51440 for details.