[SERVER-71420] Print clickable file/line in unittest failure summary Created: 16/Nov/22  Updated: 19/Oct/23

Status: Backlog
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: David Percy Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-71464 Pretty print unittest failure summary Closed
is related to SERVER-59314 Report file and line in stack traces Backlog
Assigned Teams:
Service Arch
Participants:

 Description   

When a unittest binary fails, it prints a list of tests that failed:

{"t":{"$date":"2022-11-16T19:18:37.961Z"},"s":"I",  "c":"TEST",     "id":23068,   "ctx":"main","msg":"FAILURE","attr":{"failedTestsCount":5,"failedSuitesCount":2,"failedTests":["LogicalRewriter/NotPushdownToplevel","LogicalRewriter/NotPushdownUnderLambdaSuccess","LogicalRewriter/NotPushdownUnderLambdaKeepOuterTraverse","PhysRewriter/FilterIndexingVariable","PhysRewriter/NotPushdownUnderLambdaKeepOuterTraverse"]}}

Could we change this format to make it more clickable? For example, imagine if it printed something like this:

src/mongo/db/query/optimizer/logical_rewriter_optimizer_test.cpp:1566:TEST(LogicalRewriter, NotPushdownToplevel) {
src/mongo/db/query/optimizer/logical_rewriter_optimizer_test.cpp:1628:TEST(LogicalRewriter, NotPushdownUnderLambdaSuccess) {
src/mongo/db/query/optimizer/logical_rewriter_optimizer_test.cpp:1715:TEST(LogicalRewriter, NotPushdownUnderLambdaKeepOuterTraverse) {
src/mongo/db/query/optimizer/physical_rewriter_optimizer_test.cpp:1615:TEST(PhysRewriter, FilterIndexingVariable) {
src/mongo/db/query/optimizer/physical_rewriter_optimizer_test.cpp:3948:TEST(PhysRewriter, NotPushdownToplevel) {
src/mongo/db/query/optimizer/physical_rewriter_optimizer_test.cpp:4008:TEST(PhysRewriter, NotPushdownUnderLambdaSuccess) {
src/mongo/db/query/optimizer/physical_rewriter_optimizer_test.cpp:4102:TEST(PhysRewriter, NotPushdownUnderLambdaKeepOuterTraverse) {

Then it would be easier to jump directly to each failing test.



 Comments   
Comment by Billy Donahue [ 19/Oct/23 ]

If we could try SERVER-59314 again with a more permissively-licensed dwarf library we would have clickable locations in the stack traces that are emitted on failure. Not quite the same exact thing as the request here but I'm marking them related tickets.

Comment by Jason Chan [ 17/Nov/22 ]

Thanks alexander.neben@mongodb.com, I filed SERVER-71464 to split from this ticket and assigned it to SDP.

This ticket will be about adding file and line numbers to the printed output.

Comment by Jason Chan [ 17/Nov/22 ]

Two thoughts came up from the team during triage:
1. We think there's value in adding the file and line numbers to the printed output.
2. We know there exists a tool called mrlog that takes log lines in structured log format and converts it into a more human-readable output. We believe there is a resmoke integration for mrlog so we are wondering if the SDP team considers the tool to be formally supported by some team. If so, we think the feature request here around better readability may fall under the owners of that tool. cc: alexander.neben@mongodb.com

Comment by David Percy [ 16/Nov/22 ]

As a side note, printing the names vertically is also helpful--it makes the duplicate tests more obvious.

Generated at Thu Feb 08 06:18:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.