[CXX-1103] bsoncxx test binary doesn't return test status in exit code Created: 26/Oct/16  Updated: 15/Nov/16  Resolved: 28/Oct/16

Status: Closed
Project: C++ Driver
Component/s: Testing
Affects Version/s: None
Fix Version/s: 3.0.3

Type: Bug Priority: Major - P3
Reporter: David Golden Assignee: David Golden
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

The src/bsoncxx/test/new_tests.cpp file with main doesn't return the catch test result as the exit status. To the build system it will always appear to succeed even when bsoncxx tests are failing.

Here is an example from work-in-progress test-driven development:

$ ninja test
[0/1] Running tests...
Test project /Users/david/git/mongo/cxx-driver/build
    Start 1: bson
1/3 Test #1: bson .............................   Passed    0.12 sec
    Start 2: driver
2/3 Test #2: driver ...........................   Passed    1.11 sec
    Start 3: instance
3/3 Test #3: instance .........................   Passed    0.02 sec
 
100% tests passed, 0 tests failed out of 3
 
Total Test time (real) =   1.24 sec
 
$ ./src/bsoncxx/test/test_bson
 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_bson is a Catch v1.0 b53 host application.
Run with -? for options
 
-------------------------------------------------------------------------------
[] can reach into mixed nested arrays and documents
  throws on invalid
-------------------------------------------------------------------------------
../src/bsoncxx/test/bson_get_values.cpp:129
...............................................................................
 
../src/bsoncxx/test/bson_get_values.cpp:177: FAILED:
  REQUIRE_THROWS( doc["ints"]["badKey"]["anything"] )
because no exception was thrown where one was expected:
 
===============================================================================
test cases:    59 |    58 passed | 1 failed
assertions: 22543 | 22542 passed | 1 failed

The bsoncxx test CMakeLists.txt should be modified to match the mongocxx test CMakeLists.txt file which links the main.cpp file in the catch directory.



 Comments   
Comment by Githook User [ 08/Nov/16 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CXX-1103 Make bsoncxx test binary return exit code

The bsoncxx test binary wasn't returning test status in its exit code.
This made the binary appear to be passing even when tests were failing.

This commit replaces the buggy custom main() function with the standard
one included with the catch library, just like the mongocxx tests.
Branch: 3.1-dev
https://github.com/mongodb/mongo-cxx-driver/commit/f3a121179a560de0241b066843f79ae51fa9f637

Comment by Githook User [ 28/Oct/16 ]

Author:

{u'username': u'xdg', u'name': u'David Golden', u'email': u'xdg@xdg.me'}

Message: CXX-1103 Make bsoncxx test binary return exit code

The bsoncxx test binary wasn't returning test status in its exit code.
This made the binary appear to be passing even when tests were failing.

This commit replaces the buggy custom main() function with the standard
one included with the catch library, just like the mongocxx tests.
Branch: master
https://github.com/mongodb/mongo-cxx-driver/commit/f3a121179a560de0241b066843f79ae51fa9f637

Comment by David Golden [ 26/Oct/16 ]

https://mongodbcr.appspot.com/98810005/

Generated at Wed Feb 07 22:01:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.