[SERVER-59384] Should provide ability to perform periodic DNS health checks Created: 16/Aug/21  Updated: 29/Oct/23  Resolved: 03/Feb/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0

Type: New Feature Priority: Major - P3
Reporter: Lamont Nelson Assignee: Kshitij Gupta
Resolution: Fixed Votes: 0
Labels: pm-2364-faulty-mongos
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-59383 Implement generic TCP connection heal... Closed
is depended on by SERVER-59385 Should create and consume HealthStats... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.0
Sprint: Sharding 2021-11-29, Sharding 2021-12-13, Sharding 2021-12-27, Sharding 2022-01-10, Sharding 2022-01-24, Sharding 2022-02-07
Participants:
Linked BF Score: 178
Story Points: 3

 Description   
  • test if connection can be made to configured DNS servers
  • attempt to resolve an address

We need to build a representative list of servers to test. This might be tricky of somehow we pick servers that actually do have incorrect DNS name, and we treat it as a fatal error. A simple solution:
1. we build a list of servers discovered, this may include many shards and 3 config servers. Potentially, it could be hundreds of them
2. when building the list, cap it at some number like 20, so we don't pull 1,000 servers for really large clusters
3. shuffle the list randomly and pick the first 10 servers. Every time, the list of those 10 servers to resolve will be different, thus we avoid some unfortunate combination of servers that always fail

Remember it's important to always cap the list of servers we resolve, we need to control the cost.



 Comments   
Comment by Githook User [ 14/Mar/22 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-59384: Should provide ability to perform periodic DNS health checks

(cherry picked from commit 1d8aaed7f9286ce90317d03fc1815bb58b43e31d)
Branch: v5.0
https://github.com/mongodb/mongo/commit/47833c8bd40200134206a35a137b4fc68671154f

Comment by Githook User [ 14/Mar/22 ]

Author:

{'name': 'Andrew Shuvalov', 'email': 'andrew.shuvalov@mongodb.com', 'username': 'shuvalov-mdb'}

Message: SERVER-59384 5.0 backport related changes
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/9cdfa4d6cb4d740c23b2276fc4f08c71fa7746e6

Comment by Githook User [ 03/Feb/22 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-59384: Should provide ability to perform periodic DNS health checks
Branch: master
https://github.com/mongodb/mongo/commit/1d8aaed7f9286ce90317d03fc1815bb58b43e31d

Comment by Githook User [ 03/Feb/22 ]

Author:

{'name': 'Kshitij Gupta', 'email': 'kshitij.gupta@mongodb.com', 'username': 'kshitijng'}

Message: SERVER-59384: Should provide ability to perform periodic DNS health checks.
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/8d867819126e6ce4a11fc71d9b9b0241d4106529

Generated at Thu Feb 08 05:47:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.