Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-8427

Coredump backtrace not available in the log files

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: None
    • Labels:
    • Sprint:
      Storage - Ra 2022-01-10

      Description

      Sample WT-8268, a failure on the PPC machine where python test crashed and likely generated a core file. For efficient test triage, the backtrace from the core should be available in the log itself. I thought we do that already - so this ticket might mean exploring why this particular instance of the failure doesn't show a backtrace.

      Here is the failure example:

      [2021/11/18 11:12:51.632] ======================================================================
      [2021/11/18 11:12:51.632] ERROR: test_compact03.test_compact03.test_compact03(1KB.no_truncate) (subunit.RemotedTestCase)
      [2021/11/18 11:12:51.632] test_compact03.test_compact03.test_compact03(1KB.no_truncate)
      [2021/11/18 11:12:51.632] ----------------------------------------------------------------------
      [2021/11/18 11:12:51.632] testtools.testresult.real._StringException: Traceback (most recent call last):
      [2021/11/18 11:12:51.632]   File "/data/mci/27cd1c1f6dc5a1fbc99bd8d0bb576f6d/wiredtiger/test/suite/test_compact03.py", line 156, in test_compact03
      [2021/11/18 11:12:51.632]     self.session.compact(self.uri)
      [2021/11/18 11:12:51.632] _wiredtiger.WiredTigerError: Connection timed out
      [2021/11/18 11:12:51.632] ----------------------------------------------------------------------
      [2021/11/18 11:12:51.632] Ran 7528 tests in 14271.800s
      [2021/11/18 11:12:51.707] FAILED (errors=1, skipped=271)
      

      We list the contents of WT_TEST, and for some reason we did not find it.

      [2021/11/18 11:16:49.109] Running command 'shell.exec' in "dump stderr/stdout" (step 3 of 4)
      [2021/11/18 11:16:49.140] if [ -d "WT_TEST" ]; then
      [2021/11/18 11:16:49.140] # Dump stderr/stdout contents generated by the C libraries onto console for Python tests
      [2021/11/18 11:16:49.140] find "WT_TEST" -name "std*.txt" ! -empty -exec sh -c "echo 'Contents from {}:'; cat '{}'" \;
      [2021/11/18 11:16:49.140] fi
      [2021/11/18 11:16:49.154] Finished 'shell.exec' in "dump stderr/stdout" in 44.489671ms

      Definition of done:
      A failing python test that generates a core should dump the backtrace from the core in the logs. Taking the role of the test triage baron error and backtrace should be available in the evergreen logs.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-storage-engines Backlog - Storage Engines Team
              Reporter:
              sulabh.mahajan Sulabh Mahajan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated: