Avoid complete stdout/stderr dump of massive files for Python tests

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • StorEng - Refinement Pipeline
    • None

      There are some test runs that may require a lot of work to diagnose because of a huge (possibly irrelevant) dump of a stdout.txt file.  If the stdout is verbose output, it may be tens of thousands of lines long (the case I'm looking at is 147214 lines).  This code in evergreen.yml:

               if [ -d "WT_TEST" ]; then
                # Dump stderr/stdout contents generated by the C libraries onto console for Python tests
                find "WT_TEST" -name "std*.txt" ! -empty -exec sh -c "echo 'Contents from {}:'; cat '{}'" \;
              fi
      

      will unconditionally dump those lines to the console.  The evergreen tooling is overwhelmed and either gives no feedback or captures these verbose lines.

      BTW, these verbose lines in this case come from rollback_to_stable tests, where the output is explicitly ignored within the test.  Presumably this is so that rare failures of these tests will have some good verbose info to work with.

      Anyway, we should consider changing the above find command to use "head -1000" rather than cat, or better yet, deduce that file is huge and indicate that the output is truncated.  Maybe:

      .... "echo 'Contents from {}:'; head -1000 {}; awk -- '{ if (NR==1000) {print \"{} is TRUNCATED...\";}}' {}"

       

            Assignee:
            [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: