[SERVER-13244] Secondary that enters RECOVERY mode does not reset connections Created: 18/Mar/14 Updated: 06/Dec/22 |
|
| Status: | Open |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | Needs Further Definition |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Andrew Ryder (Inactive) | Assignee: | Backlog - Replication Team |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Assigned Teams: |
Replication
|
||||
| Operating System: | ALL | ||||
| Participants: | |||||
| Description |
|
When a node changes state normally (most notably from an election) it disconnects all sockets partly to ensure the change of state is noticed. However, when a Secondary enters RECOVERY mode due to a compact command, it does not close existing sockets. Clients using secondaryPreferred continue to utilize the node as a normal secondary (operations either get blocked or return a failure due to "not primary or secondary") but new clients do not attempt to issue queries because the invalid state is detected. If the change in state means that normal client operations cannot proceed, the sockets should be reset, as is the case for elections. |
| Comments |
| Comment by Andrew Ryder (Inactive) [ 09/Apr/14 ] |
The wire protocol has no concept of node states. The insert command is grammatically identical to the getLastError command, and is encoded into BSON the same way. One of these is write-only. The other is read-only. The wire protocol cannot tell the difference. The availability of these commands is determined by the state of the instance one is connected to - nothing to do with the wire protocol. I do not see what is special about writing that does not apply to reading. The impairment of either function is of equal importance. A database that cannot be written is useless. A database that cannot be read is useless. But why not simply an error message for both conditions then? Why an error message for one, and abrupt disconnection for another? |
| Comment by Scott Hernandez (Inactive) [ 24/Mar/14 ] |
|
Drivers/clients do periodically pull/check the state of each node, and should react when the server responds with not-master/slave-ok. The reason for primary to non-primary connection dropping is because of writes (just think about the wire protocol), which is a special case and fairly important. It is not a general rule to drop connection on state changes, no – making it so would dramatically change client/system behavior. |
| Comment by Andrew Ryder (Inactive) [ 24/Mar/14 ] |
|
I agree the behavior could be addressed at the driver, but that doesn't really address my question. I want to clarify: I am inquiring as to what is special about the PRIMARY / SECONDARY transition such that it warrants dropping existing connections but which does not apply to other state changes. If the behavior to "drop all connections" when changing between SECONDARY and PRIMARY conditions is for a different reason (that is, if the change in state is incidental to the rationale behind terminating all connections) then the retention of connections through the transition to RECOVERING state can be justified as having no special significance. Right now, the SECONDARY / PRIMARY transition highlights a connection retention rule that does not appear to be consistently applied. Is there a rule being applied that I do not understand? |
| Comment by Eric Milkie [ 18/Mar/14 ] |
|
Is there a specific driver you had in mind for this behavior? I'm not sure that a blanket statement that all clients using secondaryPreferred would indefinitely produce errors when a particular node in a set is in state RECOVERING is accurate. In general, I believe this is an issue that can improved at the driver level, if it isn't already optimal. After receiving one error, it's certainly possible for the client to rescan the state of the replica set and avoid sending further read requests to a RECOVERING node. |