[SERVER-49694] On a sharded cluster, nearest or hedged reads may not be routed to a near shard replica. Created: 17/Jul/20 Updated: 29/Oct/23 Resolved: 17/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.4.1, 4.7.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Benjamin Caimano (Inactive) | Assignee: | Lamont Nelson |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Requested: |
v4.4
|
||||||||||||||||||||
| Steps To Reproduce: | I've attached a javascript test that logs the RTTs over the course of 20 pings. It shows 0 RTT reliably despite a failCommand fail point that should prevent that. |
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Linked BF Score: | 44 | ||||||||||||||||||||
| Description |
|
Issue Status as of July 28, 2020 ISSUE SUMMARY Due to an issue with reporting the round-trip time (RTT) of replica set members, on mongoS, "nearest" read preference and hedged reads do not determine the eligible nodes based on localThresholdMs. USER IMPACT On a sharded cluster, nearest reads or hedged reads are not necessarily routed to a near shard replica (that is, within 15ms round-trip time). These reads are instead dispatched to an arbitrary node, which can be more remote. WORKAROUNDS Installations with distributed sharded clusters that rely on read preference "nearest" should not upgrade to 4.4.0. Instead, upgrade to 4.4.1. AFFECTED VERSIONS This affects version 4.4.0. FIX VERSION This issue is corrected in MongoDB 4.4.1.
Original DescriptionWhen looking a substantial RTT for non-colocated hosts, I'm seeing clean .8 factor decreases on ping intervals which trends towards 0. I see that the factor is .2 from here. My conclusion is we're passing in RTT=0 for each update after the first. |
| Comments |
| Comment by Githook User [ 12/Nov/20 ] |
|
Author: {'name': 'jeff-allen-mongo', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}Message: (DOCSP-11857): |
| Comment by Githook User [ 20/Aug/20 ] |
|
Author: {'name': 'jeff-allen-mongo', 'email': 'jeffrey.allen@10gen.com', 'username': 'jeff-allen-mongo'}Message: (DOCSP-11857): |
| Comment by Jeffrey Allen [ 17/Aug/20 ] |
|
Hi lamont.nelson, we'd been documenting this ticket as a Known Issue in 4.4. Can we now remove this ticket from that section? |
| Comment by Githook User [ 17/Aug/20 ] |
|
Author: {'name': 'LaMont Nelson', 'email': 'lamont.nelson@mongodb.com', 'username': 'lamontnelson'}Message: (cherry picked from commit d09a0da1f4ccca9b087ba21bba66c88e0abd07d0) |
| Comment by Githook User [ 14/Aug/20 ] |
|
Author: {'name': 'LaMont Nelson', 'email': 'lamont.nelson@mongodb.com', 'username': 'lamontnelson'}Message: |
| Comment by Githook User [ 13/Aug/20 ] |
|
Author: {'name': 'LaMont Nelson', 'email': 'lamont.nelson@mongodb.com', 'username': 'lamontnelson'}Message: Revert " This reverts commit b2c1fa4f121fdb6cdffa924b802271d68c3367a3. |
| Comment by Githook User [ 05/Aug/20 ] |
|
Author: {'name': 'LaMont Nelson', 'email': 'lamont.nelson@mongodb.com', 'username': 'lamontnelson'}Message: |
| Comment by Lamont Nelson [ 23/Jul/20 ] |
|
Yes, Microseconds makes sense. I've started changing the code and it seems this will impact a number of files as I move through the call sites and their dependencies. I'm currently changing the types that use this field for consistency, and to preserve information. Alternatively, I could do a BFS and try to keep the changes to one level deep, but it seems that being consistent in this regard is the right thing to do since this would save any users of RemoteCommandResponse from having to instantiate their own timer (like the old RSM) if they care about the latency. If anyone feels strongly either way let me know. |
| Comment by Andy Schwerin [ 22/Jul/20 ] |
|
So, Microseconds? That's the natural return type Timer::elapsed(). |
| Comment by Lamont Nelson [ 22/Jul/20 ] |
|
This will require changing this type to a sub-millisecond type, so that the code here will have a nonzero value. |