[CDRIVER-1453] Combine mongoc code coverage reports from all variants Created: 05/Aug/16  Updated: 19/Jul/18  Resolved: 19/Jul/18

Status: Closed
Project: C Driver
Component/s: libbson, libmongoc, tests
Affects Version/s: None
Fix Version/s: Not Planned

Type: New Feature Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: Spencer Mckenney
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by CDRIVER-2749 Add a code coverage evergreen task th... Closed
Related
related to CDRIVER-729 libbson Code coverage report in Everg... Closed
Epic Link: CDRIVER-2733

 Description   

Evergreen builds on most platforms should generate coverage reports.

Can generate with gcov, lcov, and genhtml on gcc / clang platforms, don't know about MSVC yet. coveralls.io may help cross-platform issues, and / or may provide a more insightful UI.



 Comments   
Comment by Githook User [ 19/Jul/18 ]

Author:

{'username': 'spencemc', 'name': 'Spencer McKenney', 'email': 'spencermck@me.com'}

Message: CDRIVER-1453 code coverage using codecov.io
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/78d40f5744ff8748228d396b4bd9b44b41d6c2e9

Comment by Kevin Albertson [ 03/Jul/18 ]

From experimenting with both, Codecov does seem simpler to merge reports. Let's add code coverage to all of the tasks in the Valgrind (or ASAN) variant. The slowdown of code coverage won't matter there, since we already skip time sensitive tests on those variants.

Comment by A. Jesse Jiryu Davis [ 29/Jun/18 ]

Perhaps sign up for a free coverage report from coveralls.io or codecov.io (both free for open source).

With Coveralls, we'd use a Python script to convert the coverage output file to JSON for each build, and once all are complete we'd have a final step that somehow depends on all of them. The final step uses the same Python script to merge JSON files and post to Coveralls:

http://coveralls-python.readthedocs.io/en/latest/usage/multilang.html

Codecov seems simpler, it seems to merge reports for us instead of requiring us to do it in Evergreen:

https://github.com/codecov/example-c
https://docs.codecov.io/docs/merging-reports

However, I don't know with Codecov how to inform it that all testing is complete in Evergreen and it's time to finalize the report.

Comment by Hannes Magnusson [ 27/Oct/16 ]

This is a bit tricky because we have different tasks that test different things.
For example, MongoDB 2.6 codepath is never going to be triggered when running the MongoDB 3.4 task, unless we use mock servers for everything, which has the clear disadvantage of being mock, not real testing.

We maybe can do something about uploading each coverage result and fetch it before next task and see if it can be done incrementally... Not sure how much effort we want to realistically put into this.

Generated at Wed Feb 07 21:12:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.