[CDRIVER-1453] Combine mongoc code coverage reports from all variants Created: 05/Aug/16 Updated: 19/Jul/18 Resolved: 19/Jul/18 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | libbson, libmongoc, tests |
| Affects Version/s: | None |
| Fix Version/s: | Not Planned |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | A. Jesse Jiryu Davis | Assignee: | Spencer Mckenney |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Epic Link: | CDRIVER-2733 | ||||||||||||||||
| Description |
|
Evergreen builds on most platforms should generate coverage reports. Can generate with gcov, lcov, and genhtml on gcc / clang platforms, don't know about MSVC yet. coveralls.io may help cross-platform issues, and / or may provide a more insightful UI. |
| Comments |
| Comment by Githook User [ 19/Jul/18 ] |
|
Author: {'username': 'spencemc', 'name': 'Spencer McKenney', 'email': 'spencermck@me.com'}Message: |
| Comment by Kevin Albertson [ 03/Jul/18 ] |
|
From experimenting with both, Codecov does seem simpler to merge reports. Let's add code coverage to all of the tasks in the Valgrind (or ASAN) variant. The slowdown of code coverage won't matter there, since we already skip time sensitive tests on those variants. |
| Comment by A. Jesse Jiryu Davis [ 29/Jun/18 ] |
|
Perhaps sign up for a free coverage report from coveralls.io or codecov.io (both free for open source). With Coveralls, we'd use a Python script to convert the coverage output file to JSON for each build, and once all are complete we'd have a final step that somehow depends on all of them. The final step uses the same Python script to merge JSON files and post to Coveralls: http://coveralls-python.readthedocs.io/en/latest/usage/multilang.html Codecov seems simpler, it seems to merge reports for us instead of requiring us to do it in Evergreen: https://github.com/codecov/example-c However, I don't know with Codecov how to inform it that all testing is complete in Evergreen and it's time to finalize the report. |
| Comment by Hannes Magnusson [ 27/Oct/16 ] |
|
This is a bit tricky because we have different tasks that test different things. We maybe can do something about uploading each coverage result and fetch it before next task and see if it can be done incrementally... Not sure how much effort we want to realistically put into this. |