In local execution, this is possible by setting --test_arg=–testTimeout=N. Resmoke will find gdb at /opt/mongodbtoolchain/v5/bin/gdb.

In remote execution, we need to bring our own gdb, since there is not one installed in the container image. Even if including the toolchain gdb in the action's runfiles, the attaching to an arbitrary process is blocked by restricted privileges in the container. Attempting to do so will result in a "ptrace: Operation not permitted." In order to allow this, we need to set "dockerAddCapabilities": "SYS_PTRACE" as an exec_properties for the action, and either grant the non-root user permission to ptrace (somehow?) or set cap_sys_ptrace on the debugger that we plan to use.

Ref: https://man7.org/linux/man-pages/man2/ptrace.2.html#:~:text=access%20is%20allowed.-,/proc/sys/kernel/yama/ptrace_scope,-On%20systems%20with

This is not the right way to do it, but this in the remote execution dockerfile gets this working:

RUN apt-get install gdb

RUN setcap cap_sys_ptrace=iep /bin/gdb

Then the test action can use this gdb to attach to ANY process. Hopefully there is a better alternative.

What I thought would work, and does in the local docker spawn stragegy (but not remote) is setting PR_SET_PTRACER_ANY from each process we potentially want to attach to and dump. Described in /etc/sysctl.d/10-ptrace.conf. In python:

def set_ptracer_any():
    try:
        libc = ctypes.CDLL("libc.so.6")
    except OSError:
        sys.exit(1)
    PR_SET_PTRACER = 0x59616D61  # Defined in <linux/prctl.h>
    PR_SET_PTRACER_ANY = 1 << 0  # Allow any process to trace
    result = libc.prctl(PR_SET_PTRACER, PR_SET_PTRACER_ANY, 0, 0, 0)
    if result != 0:
        sys.exit(1)

proc = subprocess.Popen(["sleep", "5"], preexec_fn=set_ptracer_any)

subprocess.run(["gdb", "--pid", str(proc.pid), ...snip...])

Ref: https://man7.org/linux/man-pages/man2/PR_SET_PTRACER.2const.html