[SERVER-43258] {shutdown:1} command (always?) fails Created: 10/Sep/19 Updated: 06/Dec/22 Resolved: 24/Feb/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Minor - P4 |
| Reporter: | Oleg Pudeyev (Inactive) | Assignee: | Backlog - Service Architecture |
| Resolution: | Won't Do | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Service Arch
|
| Sprint: | Service Arch 2019-09-23, Service Arch 2019-10-07 |
| Participants: |
| Description |
|
When a client sends {shutdown:1}admin command, the server appears to close the connection on which the command was sent prior to sending a response. Shell test:
Ruby driver test:
Given that the command always fails, it is difficult/impossible for the application trying to shut down a server to verify that the server initiated the shutdown command. The command can fail due to being issued against the wrong db for example:
This is especially problematic when scripting `mongo` from the Unix shell since simply relying on the exit status of the command does not provide any guarantees as to the success/failure for {shutdown:1}. The driver can inspect precise error message returned and act accordingly, but this requires a fair amount of effort and does not help if there was in fact a network problem during the execution of the command. The server should send a successful response to the command and then close the connection (or perform other shutdown-related activities). |
| Comments |
| Comment by Lauren Lewis (Inactive) [ 24/Feb/22 ] | |||||||||||||||||||||
|
We haven’t heard back from you for at least one calendar year, so this issue is being closed. If this is still an issue for you, please provide additional information and we will reopen the ticket. | |||||||||||||||||||||
| Comment by Oleg Pudeyev (Inactive) [ 13/Sep/19 ] | |||||||||||||||||||||
|
To clarify, I think it is perfectly ok to close (or start closing) the connection on the server after sending the response to the shutdown command. My impression is right now the connection on which shutdown command is received is closed without a response being sent first. > Would that best effort response actually be useful to you? Yes, this is what I had in mind I think. | |||||||||||||||||||||
| Comment by Mira Carey [ 13/Sep/19 ] | |||||||||||||||||||||
The problem is that there's no way with tcp (up at the stream level) to know that the other side has received any bytes you've sent. You need some kind of signal that indicates that the client has received your response. Either closing the socket, or a reply sent to you. In the absence of that, you have to rely on timeouts. I think this is a thing we could tack on, because there's already a certain amount of extra time after we've decided to enter terminal shutdown before the process actually goes down, but I'd still constrain that amount of time we waited. I.e. I'd send the reply, half close the socket and then leave it open until actual process death. I think that would make the user experience a bit cleaner most of the time, but I still wouldn't be able to guarantee you always get that response (without changing the contract with clients). Would that best effort response actually be useful to you? | |||||||||||||||||||||
| Comment by Oleg Pudeyev (Inactive) [ 13/Sep/19 ] | |||||||||||||||||||||
|
mysql seems to be able to do it:
Given that the shutdown command was just received on a connection, in most cases the connection should be operational to send the response on and close it I would think? | |||||||||||||||||||||
| Comment by Mira Carey [ 13/Sep/19 ] | |||||||||||||||||||||
Part of the problem is that we don't have any way to do that reliably. We can ::send() a response, but all that does is put it on the outbound write queue, it doesn't say anything about when the other side will receive it. A ::send() followed shortly thereafter by a ::shutdown() doesn't turn into a received message. The only reliable thing would be to make the flow:
Doing that would effectively prevent the shutdown from proceeding until the client went away For something unreliable, we could add in a sleep, but do you think it's worthwhile if it isn't reliable? It's also worth noting that any waiting would effectively make shutdown take longer for all existing users in order to enable a use case which doesn't currently work. | |||||||||||||||||||||
| Comment by Oleg Pudeyev (Inactive) [ 12/Sep/19 ] | |||||||||||||||||||||
|
It is my understanding that https://jira.mongodb.org/browse/SERVER-5467 essentially proposes to work around the server always failing the command in the shell. While this will fix the shell, it will not do anything for drivers and thus for any applications using the drivers to interact with the server. However, if this ticket is addressed then https://jira.mongodb.org/browse/SERVER-5467 would become redundant, thus I think https://jira.mongodb.org/browse/SERVER-5467 can be closed as a duplicate of this ticket. | |||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 11/Sep/19 ] | |||||||||||||||||||||
|
I believe this falls under |