[SERVER-46028] non-interactive mongo shell tries to write history file before aborting Created: 07/Feb/20  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Trivial - P5
Reporter: Lungang Fang Assignee: Backlog - Server Tooling and Methods (STM) (Inactive)
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Tooling & Methods
Operating System: ALL
Steps To Reproduce:
  1. Spin up a mongod
  2. Connect to it using mongo shell 3.6 or above:

    mongo --eval 'while(true) {printjson(new Date()); sleep(2000)}'
    

  3. Press Ctrl-c to interrupt
  4. You'll see an error message like the following:

    2020-02-07T16:00:22.482+1100 E -        [main] Error saving history file: FileOpenFailed: Unable to open() file : No such file or directory
    2020-02-07T16:00:22.482+1100 I CONTROL  [main] shutting down with code:0
    

Participants:
Case:

 Description   

Hi,

If I understand the following source code snippet correctly, the mongo shell will not write the history file if the process is non-interactive but ::shellHistoryDone() is registered as a shutdown task at the very beginning of _main.

//  mongo/src/mongo/shell/dbshell.cpp
int _main(int argc, char* argv[], char** envp) {
    registerShutdownTask([] {
        ...
        ::killOps();
        ::shellHistoryDone();
    });
    ...
    if (shellGlobalParams.files.size() == 0 && shellGlobalParams.script.empty())
        shellGlobalParams.runShell = true;
    ...
    if (shellGlobalParams.runShell) {
        ...
        shellHistoryInit();
        while (1) {...}
 
        shellHistoryDone();
    }
    ...
}

As a result: if a non-interactive mongo shell aborts, it will try to save history and report error [main] Error saving history file: FileOpenFailed: Unable to open() file : No such file or directory because the history file name is empty.

I understand that mongo shell will continue working and this error message can actually serve as an indication that the non-interactive mongo shell aborts. However, I found some were confused by the error. Therefore, I'd suggest either move registering ::shellHistoryDone() as a shutdown task into the interactive loop if (shellGlobalParams.runShell) or skip linenoiseHistorySave in shellHistoryDone() if the history file path is empty.

Regards,
Lungang



 Comments   
Comment by Steven Vannelli [ 10/May/22 ]

Moving this ticket to the Backlog and removing the "Backlog" fixVersion as per our latest policy for using fixVersions.

Comment by Kevin Pulo [ 10/Feb/20 ]

Excellent clarification, thanks!

Comment by Lungang Fang [ 10/Feb/20 ]

Hi Kev,

Thanks for looking into this issue.

Firstly, my apologies for the confusion. The behaviour I reported was for 4.0.14 but the source code I referred to is 4.2.

Is this a duplicate of SERVER-31077?

There are related but not exactly the same. The issue reported here is: when a mongo shell decides to abort, it tries to write the history file even if it is a non-interactive session. As a result, in 4.0, it will hit SERVER-31077 because the path of history file is empty.

I can't repro with 4.2.0 or later, only 4.0.16 or earlier — can you confirm you see the same behaviour? If so then it's likely been fixed in the current code.

In 4.2, I can confirm that the issue still exists but the symptom is different because of changes introduced in commit 159eba44d7a.

In branch 4.0, the history file path is initialized in shellHistoryInit(), which is only called for interactive sessions (see below). Therefore, non-interactive mongo shell sessions will get empty history file path and hence report the error I reported earlier.

void shellHistoryInit() {
    ...
    const char* h = shell_utils::getUserDir();
    if (h)
        ss << h << "/";
    ss << ".dbshell";
    historyFile = ss.str();
    ...
}
 
void shellHistoryDone() {
    Status res = linenoiseHistorySave(historyFile.c_str());
    ...
}
 
int _main(int argc, char* argv[], char** envp) {
    ...
    if (shellGlobalParams.runShell) {
        ...
        shellHistoryInit();
        while (1) {...}
    ...
}

In comparison, after the commit 159eba44d7a, the history file path is constructed from the user's home dir on the fly:

boost::filesystem::path mongo::shell_utils::getHistoryFilePath() {
    static const auto& historyFile = *new boost::filesystem::path(getUserDir() / ".dbshell");
 
    return historyFile;
}
 
void shellHistoryDone() {
    Status res = linenoiseHistorySave(shell_utils::getHistoryFilePath().string().c_str());
    ...
}

As a result, if a non-interactive mongo shell session aborts, it will able to get the history file path and update the history file. However, in my opinion, non-interactive mongo shell sessions should not update the history file in any circumstances. Even worse, according to my tests with 4.2 (see below), a non-interactive mongo shell session will truncated the history file should it abort. I believe this is because shellHistoryInit() is not called for non-interactive sessions.

$ mongo --version
MongoDB shell version v4.2.0
git version: a4b751dcf51dd249c5865812b390cfd1c0129c30
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64
 
$ cat ~/.dbshell
db.foo.find()
 
$ mongo --eval 'printjson(new Date())' --quiet   <=== completes execution successfully
ISODate("2020-02-09T23:53:19.746Z")
 
$ cat ~/.dbshell 
db.foo.find()                                     <=== 'printjson' is *not* appended to the history file, which is desired behaviour
 
$ mongo --eval 'while(true) {printjson(new Date()); sleep(2000)}' --quiet       
ISODate("2020-02-09T23:48:04.755Z")
ISODate("2020-02-09T23:48:06.755Z")
ISODate("2020-02-09T23:48:08.757Z")
^C2020-02-10T10:48:09.045+1100 I  CONTROL  [main] shutting down with code:0     <=== interrupted by Ctrl-c
 
$ cat ~/.dbshell                                  <=== the history file is truncated
 
$

Running the tests again with the file permission of the history file changed to read only (see below), we can see clearly that:
a) The normal call flow of non-interactive mongo shell sessions does not write the history file (desired behaviour).
b) Should it decide to abort, the mongo shell does try to write the history file.

$ grep mongod /etc/passwd
mongod:x:996:994:mongod:/var/lib/mongo:/bin/false
 
$ ls -l /var/lib/mongo/.dbshell
-r-------- 1 mongod mongod 0 Feb 10 00:12 /var/lib/mongo/.dbshell
 
$ sudo -u mongod mongo --eval 'while(true) {printjson(new Date()); sleep(2000)}' --quiet
ISODate("2020-02-10T00:15:26.554Z")
ISODate("2020-02-10T00:15:28.555Z")
^C2020-02-10T00:15:29.196+0000 E  -        [main] Error saving history file: FileOpenFailed: Unable to open() file /var/lib/mongo/.dbshell: Permission denied
2020-02-10T00:15:29.196+0000 I  CONTROL  [main] shutting down with code:0
 
$ sudo -u mongod mongo --eval 'printjson(new Date())' --quiet
ISODate("2020-02-10T00:16:16.514Z")
 
$

In summary, both 4.0 and 4.2 non-interactive mongo shell sessions try to write the shell history file, which is not desired and causes different issues in different branches. Since it is common to not record commands in non-interactive sessions, I believe the best (and least surprising) fix for this issue is to not register shellHistoryDone() as a shutdown task for non-interactive mongo shell sessions.

Best Regards,
Lungang

Comment by Kevin Pulo [ 07/Feb/20 ]

Is this a duplicate of SERVER-31077? Also, I can't repro with 4.2.0 or later, only 4.0.16 or earlier — can you confirm you see the same behaviour? If so then it's likely been fixed in the current code.

Generated at Thu Feb 08 05:10:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.