[SERVER-65700] list-append's Evergreen report generator masks failures if a single iteration succeeds Created: 15/Apr/22  Updated: 29/Oct/23  Resolved: 31/May/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 6.1.0-rc0, 6.0.6

Type: Bug Priority: Major - P3
Reporter: Richard Samuels (Inactive) Assignee: Richard Samuels (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
is related to SERVER-65539 jepsen_list-append failures do not up... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0
Sprint: Dev Platform 2022-04-18, Dev Platform 2022-05-02, Dev Platform 2022-05-16, Dev Platform 2022-05-30
Participants:

 Description   

The list-append workload, unlike all other jepsen workloads, runs 30 times (as opposed to 1). jepsen_report.sh erroneously flags a single test's success as success for all 30 tests.

This task shows a test that had a failure masked by this bug.

Fix jepsen_report.sh so that one failure causes the whole task to fail.



 Comments   
Comment by Githook User [ 21/Apr/23 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-65700 list-append's Evergreen report generator masks failures if a single iteration succeeds

(cherry picked from commit 48d37ac38656aab14427a985653e221b6ed78550)

Also adds the simple_report.py module to satisfy the dependency of the
jepsen_report.py module. This version of the simple_report.py module
includes the changes from 09173712c719030f4c1ee499578e25d10fbc536f,
425eb62e4440d2de191ce336ae1557424beda449, and
8d463a6789d68e2fc39b93346152e1b4a626e14a.
Branch: v6.0
https://github.com/mongodb/mongo/commit/aec19f4f9406979f7f7a365842c35ff8683e159b

Comment by Githook User [ 24/May/22 ]

Author:

{'name': 'Richard Samuels', 'email': 'richard.l.samuels@gmail.com', 'username': 'richardsamuels'}

Message: SERVER-65700 list-append's Evergreen report generator masks failures if a single iteration succeeds
Branch: master
https://github.com/mongodb/mongo/commit/48d37ac38656aab14427a985653e221b6ed78550

Comment by Robert Guo (Inactive) [ 16/May/22 ]

Will do!

Comment by Iryna Zhuravlova [ 16/May/22 ]

hey robert.guo@mongodb.com, can you review this ticket

Generated at Thu Feb 08 06:03:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.