[SERVER-36086] Task tries to execute batch commands inside lldb Created: 18/May/18  Updated: 06/Dec/22  Resolved: 28/Jan/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Randolph Tan Assignee: Backlog - Server Tooling and Methods (STM) (Inactive)
Resolution: Won't Fix Votes: 0
Labels: stm, tig-evgconfig, tig-hanganalyzer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Tooling & Methods
Operating System: ALL
Participants:

 Description   

Saw this in one of the stacktraces:

(lldb) process save-core dump_mongo.15796.core
error: Failed to save core file for process: process doesn't support getting memory region info
(lldb) # Call hang_analyzer.py script for tasks that are running remote mongo processes
(lldb) if [ -n "" ]; then
error: 'if' is not a valid command.
error: Unrecognized command 'if'.
(lldb)   core_ext=core
error: 'core_ext' is not a valid command.
error: Unrecognized command 'core_ext=core'.
(lldb)   if [ "Windows_NT" = "$OS" ]; then
error: 'if' is not a valid command.
error: Unrecognized command 'if'.
(lldb)     core_ext=mdmp
error: 'core_ext' is not a valid command.
error: Unrecognized command 'core_ext=mdmp'.
(lldb)   fi
Ambiguous command 'fi'. Possible matches:
	file
	finish
ambiguous command 'fi'. Possible completions:
	file
	finish
(lldb)   ssh_connection_options=" -o GSSAPIAuthentication=no -o CheckHostIP=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=20 -o ConnectionAttempts=20"
error: 'ssh_connection_options' is not a valid command.
error: Unrecognized command 'ssh_connection_options= -o GSSAPIAuthentication=no -o CheckHostIP=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ConnectTimeout=20 -o ConnectionAttempts=20'.
(lldb)   # buildscripts must be installed in  on the remote host.
(lldb)   remote_dir=.
error: 'remote_dir' is not a valid command.
error: Unrecognized command 'remote_dir=.'.
(lldb)   # Copy mongoDB debug symbols to the remote host.
(lldb)   debug_files=$(ls *.debug *.dSYM *.pdb 2> /dev/null)
error: 'debug_files' is not a valid command.
error: Unrecognized command 'debug_files=$(ls'.
(lldb)   for debug_file in $debug_files
error: 'for' is not a valid command.
error: Unrecognized command 'for'.
(lldb)   do
error: invalid thread
(lldb)     file_param="$file_param --file $debug_file"
error: 'file_param' is not a valid command.
error: Unrecognized command 'file_param=$file_param --file $debug_file'.
(lldb)   done
error: 'done' is not a valid command.
error: Unrecognized command 'done'.
(lldb)   if [ ! -z "$file_param" ]; then
error: 'if' is not a valid command.
error: Unrecognized command 'if'.
(lldb)     $python buildscripts/remote_operations.py          \
error: '$python' is not a valid command.
error: Unrecognized command '$python'.
(lldb)       --verbose                                        \
error: '--verbose' is not a valid command.
error: Unrecognized command '--verbose'.
(lldb)       --userHost $USER@           \
error: '--userHost' is not a valid command.
error: Unrecognized command '--userHost'.
(lldb)       --operation "copy_to"                            \
error: '--operation' is not a valid command.
error: Unrecognized command '--operation'.
(lldb)       --sshConnectionOptions "$ssh_connection_options" \
error: '--sshConnectionOptions' is not a valid command.
error: Unrecognized command '--sshConnectionOptions'.
(lldb)       --retries 10                         \
error: '--retries' is not a valid command.
error: Unrecognized command '--retries'.
(lldb)       $file_param                                      \
error: '$file_param' is not a valid command.
error: Unrecognized command '$file_param'.
(lldb)       --remoteDir $remote_dir
error: '--remoteDir' is not a valid command.
error: Unrecognized command '--remoteDir'.
(lldb)   fi
Ambiguous command 'fi'. Possible matches:
	file
	finish
ambiguous command 'fi'. Possible completions:
	file
	finish
(lldb)   # Activate virtualenv on remote host. The virtualenv bin_dir is different for Linux and
(lldb)   # Windows.
(lldb)   bin_dir=$(find $VIRTUAL_ENV -name activate | sed -e "s,$VIRTUAL_ENV,,;s,activate,,;s,/,,g")
error: 'bin_dir' is not a valid command.
error: Unrecognized command 'bin_dir=$(find'.
(lldb)   cmds=". venv/$bin_dir/activate"
error: 'cmds' is not a valid command.
error: Unrecognized command 'cmds=. venv/$bin_dir/activate'.
(lldb)   # In the 'cmds' variable we pass to remote host, use 'python' instead of '$python' since
(lldb)   # we don't want to evaluate the local python variable, but instead pass the python string
(lldb)   # so the remote host will use the right python when the virtualenv is sourced.
(lldb)   cmds="$cmds; cd "
error: 'cmds' is not a valid command.
error: Unrecognized command 'cmds=$cmds; cd '.
(lldb)   cmds="$cmds; PATH=\"/opt/mongodbtoolchain/gdb/bin:\$PATH\" python buildscripts/hang_analyzer.py $hang_analyzer_option"
error: 'cmds' is not a valid command.
error: Unrecognized command 'cmds=$cmds; PATH="/opt/mongodbtoolchain/gdb/bin:\$PATH" python buildscripts/hang_analyzer.py $hang_analyzer_option'.
(lldb)   $python buildscripts/remote_operations.py          \
error: '$python' is not a valid command.
error: Unrecognized command '$python'.
(lldb)     --verbose                                        \
error: '--verbose' is not a valid command.
error: Unrecognized command '--verbose'.
(lldb)     --userHost $USER@           \
error: '--userHost' is not a valid command.
error: Unrecognized command '--userHost'.
(lldb)     --sshConnectionOptions "$ssh_connection_options" \
error: '--sshConnectionOptions' is not a valid command.
error: Unrecognized command '--sshConnectionOptions'.
(lldb)     --retries 10                         \
error: '--retries' is not a valid command.
error: Unrecognized command '--retries'.
(lldb)     --commands "$cmds"
error: '--commands' is not a valid command.
error: Unrecognized command '--commands'.
(lldb)   $python buildscripts/remote_operations.py          \
error: '$python' is not a valid command.
error: Unrecognized command '$python'.
(lldb)     --verbose                                        \
error: '--verbose' is not a valid command.
error: Unrecognized command '--verbose'.
(lldb)     --userHost $USER@           \
error: '--userHost' is not a valid command.
error: Unrecognized command '--userHost'.
(lldb)     --operation "copy_from"                          \
error: '--operation' is not a valid command.
error: Unrecognized command '--operation'.
(lldb)     --sshConnectionOptions "$ssh_connection_options" \
error: '--sshConnectionOptions' is not a valid command.
error: Unrecognized command '--sshConnectionOptions'.
(lldb)     --retries 10                         \
error: '--retries' is not a valid command.
error: Unrecognized command '--retries'.
(lldb)     --file "$remote_dir/debugger*.*"                 \
error: '--file' is not a valid command.
error: Unrecognized command '--file'.
(lldb)     --file "$remote_dir/*.$core_ext"
error: '--file' is not a valid command.
error: Unrecognized command '--file'.
(lldb) fi
Ambiguous command 'fi'. Possible matches:
	file
	finish
ambiguous command 'fi'. Possible completions:
	file
	finish

this is downloaded from this page:
https://evergreen.mongodb.com/task/mongodb_mongo_master_osx_1010_ssl_concurrency_sharded_replication_with_balancer_cf339b8a8d8708e8b28747fe0cafee7cc79fe9a6_18_05_16_00_03_26/0



 Comments   
Comment by Mark Benvenuto [ 11/Jul/18 ]

Yes, we should close stdin when we execute lldb.

Comment by Max Hirschhorn [ 11/Jul/18 ]

I think this is just an example of lldb getting confused about how it should be processing commands. It's being run as

[2018/05/16 07:31:57.502] ['/usr/bin/lldb', '--source', '/var/folders/hf/_65m8c493xb1spz67_3kd0200000gm/T/tmpP2n5Se']

where that file contains the following commands:

[2018/05/16 07:31:57.496] ['cat', '/var/folders/hf/_65m8c493xb1spz67_3kd0200000gm/T/tmpP2n5Se']
[2018/05/16 07:31:57.502] attach -p 15796
[2018/05/16 07:31:57.502] target modules list
[2018/05/16 07:31:57.502] thread backtrace all
[2018/05/16 07:31:57.502] process save-core dump_mongo.15796.core
[2018/05/16 07:31:57.502] settings set interpreter.prompt-on-quit false
[2018/05/16 07:31:57.502] quit

and yet it tries to process other commands by reading the stdin it inherits from the parent process. mark.benvenuto, should we just explicitly pass /dev/null as the stdin for lldb?

Generated at Thu Feb 08 04:41:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.