[SERVER-58274] std::exception::what(): Invalid argument Created: 06/Jul/21  Updated: 01/Jun/22  Resolved: 21/Jul/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Sourabh Ghosh Assignee: Dmitry Agranat
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2021-07-06-16-43-01-782.png     File mongod.log.bz2     HTML File output    
Operating System: ALL
Participants:

 Description   

Hello Team,
recently we have faced some issue with one of our mongod instance wherein it got killed abruptly with an error 2021-07-05T11:08:12.757-0500 F - [conn4333398] std::exception::what(): Invalid argument
Actual exception type: std::system_error
0x56419b5dc381 0x56419b5dbcc6 0x7fd566ee1ae6 0x7fd566ee1b21 0x7fd566ee1d54 0x7fd566edda23 0x56419b5cf753 0x564199cfc3d4 0x564199cfb11f 0x56419afc74e6 0x56419afcc741 0x564199e841ce 0x564199e868b4 0x564199e8764d 0x564199e75c6c 0x564199e8149c 0x564199e7ebe7 0x564199e80535 0x56419acf49f2 0x564199e7be6d 0x564199e7d57b 0x564199e7e036 0x564199e7ebab 0x564199e80535 0x56419acf4e77 0x56419b35b685 0x56419b35b6e4 0x7fd5666816db 0x7fd5663aa88f

not sure on the error part what exactly caused the issue, i have shared the error log for the day wherein the incident took place at 2021-07-05T11:08:12.757-0500  EST

the below error we got from syslog



 Comments   
Comment by Dmitry Agranat [ 13/Jul/21 ]

Hi sourabh.ghosh@visiblealpha.com, based on the backtrace and the symbolized output from the percona binaries, it is not clear what the issue is beyond some relation to a deprecated command. If you are able to reproduce this with MongoDB binaries, we will be happy to take a look.

Comment by Aayushi Mangal [ 09/Jul/21 ]

Hi Dmitry,

I am replying on behalf of my colleague Sourabh, since we were not sure about the cause of this crash, we will not be able to reproduce the case with MongoDB binaries. 

To understand what exactly causes this failure, we tried to read backtrace part and found some functions involved. Are these anyway relevant to this failure, let us help to understand please, so we will try to reproduce this case on both the binaries.

Also I am suspecting does any minor version 4.2.8 and 4.2.12 can be culprit here, any deprecated command or something, as we will not be able to trace any such command.

we are using this mongosymb.py here:

python3 mongosymb.py --symbolizer-path=/usr/bin/llvm-symbolizer-12 /home/aayushi/mongodb-linux-x86_64-ubuntu1804-4.2.8/mongod.debug < stacktrace.json --output-format=json > output

output

{ "fn": "mongo::User::hasRole(mongo::RoleName const&) const", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/auth/user.cpp", "column": 24, "line": 85 }

{ "fn": "mongo::RoleGraph::removePrivilegeFromRole(mongo::RoleName const&, mongo::Privilege const&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/auth/role_graph.cpp", "column": 43, "line": 323 } { "fn": "mongo::CmdCreateRole::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 19, "line": 1468 } { "fn": "AuthzLockGuard", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 5, "line": 587 }

,

 

{ "fn": "mongo::CmdDropAllRolesFromDatabase::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 19, "line": 2109 }

 

{ "fn": "mongo::CmdDropAllRolesFromDatabase::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 13, "line": 2152 }

 

"symbinfo": [

{ "fn": "mongo::(anonymous namespace)::updateRoleDocument(mongo::OperationContext*, mongo::RoleName const&, mongo::BSONObj const&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 100, "line": 431 }

 

{ "fn": "mongo::CmdUpdateRole::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 61, "line": 1547 }

 

"fn": "mongo::CmdRevokeRolesFromRole::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)",
"file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp",
"column": 50,
"line": 1913
}

{ "fn": "mongo::CmdRevokePrivilegesFromRole::run(mongo::OperationContext*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::BSONObj const&, mongo::BSONObjBuilder&)", "file": "/data/mci/820e455d26d8d25467db6a844e898d65/src/src/mongo/db/commands/user_management_commands.cpp", "column": 9, "line": 1751 }

]

 

 

 

Comment by Dmitry Agranat [ 08/Jul/21 ]

Hi sourabh.ghosh@visiblealpha.com, it is not immediately clear what happened. Do you see the same issue when running with binaries provided by MongoDB?

Generated at Thu Feb 08 05:44:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.