[CXX-2385] Improve unified test driver error reporting Created: 12/Oct/21  Updated: 08/Feb/23

Status: Backlog
Project: C++ Driver
Component/s: Testing, Unified Test Runner
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Unknown
Reporter: Jesse Williamson (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Epic Link: CXX Driver Testing (Evergreen)

 Description   

The unified test driver could do a better job attaching more information to error messages, enabling developers to much more quickly resolve some issues.

For example, in "mongocxx/test/spec/unified_tests/runner.cpp", run_unified_format_tests_in_env_dir() uses CAPTURE() and REQUIRE() as error reporting facilities. This does not provide information about what the error condition /was doing/: looking up an enviornment variable? Trying to find a test_files.txt file? Attaching even a small amount of further information to the thrown exceptions (used by Catch2) would really help.

This ticket is meant to represent incremental improvement, not a final or complete list of projects.



 Comments   
Comment by Jesse Williamson (Inactive) [ 03/Nov/21 ]

std::cout is called inside of Catch2 tests in a few places, this should be made "Catch-y", as the Catch2 library is free to play with the streams.

Comment by Jesse Williamson (Inactive) [ 29/Oct/21 ]

Here's another cleanup item: we should probably just filter ignored events before we actually even dispatch (i.e. should_ignore() should
not need to appear in boilerplate each time).

void apm_checker::set_command_started(options::apm& apm) {
using namespace bsoncxx::builder::basic;

apm.on_command_started([&](const events::command_started_event& event) {

// JFW: this snippet appears so often that maybe where we dispatch events is actually
// where we should filter this-- and not even call something we should ignore, to begin
// with (see all dispatch functions):
if (should_ignore(event.command_name()))

{ return; }
Comment by Jesse Williamson (Inactive) [ 27/Oct/21 ]

While we're in here, it may be well worth giving some attention to things like the duplicated functions that don't share quite the same parameters or namespaces (such as lookup_write_concern(), duplicated in a couple places in the project AND ALSO with competing implementations like get_write_concern(), which serves a suspiciously similar purpose); my bet is that the implementations may also be a bit different. It's a good idea to make them consistent.

Generated at Wed Feb 07 22:05:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.