Our current implementation of choosing a sync source for initial sync as well as steady state sync could be improved to include preferences provided by the user for situations including:
- not wanting to sync from the primary to mitigate impact to production applications that are targeting the primary
- preference to sync from a node on the same LAN vs having multiple nodes make the trip across the WAN (even if ping times would dictate otherwise)
- syncing from a specific node based on specific user preference
If the preference is not specified or can not be accommodated then we should fall back to choosing based on latency and currency as we do today.
It would be helpful for the specification of a sync source preference to be part of the replica set configuration so it is easily noticed and easily managed. The log should reflect the preference and the reason it was overridden when need be.