Uploaded image for project: 'Evergreen'
  1. Evergreen
  2. EVG-14391

Add a view for a summary of a patch build grouping by failure features



    • New Feature
    • Status: Closed
    • Major - P3
    • Resolution: Won't Do
    • None
    • None
    • ui


      Often times when a patch build is submitted there will be a handful of different problems, but it's hard to see without digging in deeper. It would be cool if there were a tab which had some information like "7 tasks failed, 3 of them with this error and 4 with other errors." The details are fuzzy, but as inspiration we do something similar in our concurrency tests. Here's some sample output from a random BF I found:

      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.433+0000 Error: 2 exceptions were thrown, 2 of which were unique:
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.433+0000 
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.433+0000         1 thread with tids [67] threw
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.433+0000         Foreground jstests/concurrency/fsm_workloads/multi_statement_transaction_atomicity_isolation_repeated_reads.js
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.433+0000         Error: BSONElement: bad type 109 @ 0x7f4685648ec1 at offset 1 in block: 0x73 0x6d 0x63 0x6f 0x6c 0x6c 0x36 0x0 0x12 0x74 0x78 0x6e 0x4e 0x75 0x6d 0x0 0x2 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x3 0x32 0x0 0x45 0x0 0x0 0x0
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000 
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         DBQuery.prototype.next@src/mongo/shell/query.js:307:1
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         DBQuery.prototype.toArray@src/mongo/shell/query.js:342:16
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         repeatedRead/<@jstests/concurrency/fsm_workloads/multi_statement_transaction_atomicity_isolation_repeated_reads.js:25:40
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         withTxnAndAutoRetry@jstests/concurrency/fsm_workload_helpers/auto_retry_transaction.js:130:17
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         repeatedRead@jstests/concurrency/fsm_workloads/multi_statement_transaction_atomicity_isolation_repeated_reads.js:22:9
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         runFSM@jstests/concurrency/fsm_libs/fsm.js:122:17
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         @eval:8:9
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         main@jstests/concurrency/fsm_libs/worker_thread.js:223:17
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         @eval:5:12
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         @eval:3:24
      [parallel_fsm_workload_test:c4e599e8c24eab6436fbdae8511838bc] 2020-05-11T14:37:04.434+0000         _threadStartWrapper@:26:16

      This would take some thinking about UX and I wouldn't expect it to be perfect in terms of grouping things together, but the feature extraction which happens for generating BFs/BFGs and the de-duplication logic we have built there should come in handy.

      The impact on developer productivity could be large. I don't think it is uncommon for engineers to diagnose and fix one problem with their patch, only to put up another patch build and find there was another problem lurking in the first patch which they didn't notice because the first failure happened some large number of times and that's all that they noticed.

      I could also save a lot of clicking through to get a high level view of what's going on with your patch.


        Issue Links



              backlog-server-evg Backlog - Evergreen Team
              charlie.swanson@mongodb.com Charlie Swanson
              0 Vote for this issue
              6 Start watching this issue