[CDRIVER-1565] JSON output from mongo-c-driver-performance suite for Evergreen perf stats Created: 27/Sep/16 Updated: 03/May/17 Resolved: 05/Oct/16 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | TBD |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | A. Jesse Jiryu Davis | Assignee: | A. Jesse Jiryu Davis |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Epic Link: | BUILD-17113 | ||||||||
| Description |
|
The C Driver's performance benchmark is here: https://github.com/mongodb/mongo-c-driver-performance Right now it outputs comma-delimited stats. Update it to output JSON to a file, in the format consumed by Evergreen's "performance visualization" plugin. |
| Comments |
| Comment by Brian McCarthy [ 04/Oct/16 ] | ||||||||||||||||||||||||||||||||
|
Just tested it out in evergreen and it works. | ||||||||||||||||||||||||||||||||
| Comment by Brian McCarthy [ 04/Oct/16 ] | ||||||||||||||||||||||||||||||||
|
I just ran a couple of the tests and it seems to be formatting correctly:
I'm going to run a full test in the perf dashboard. | ||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 04/Oct/16 ] | ||||||||||||||||||||||||||||||||
|
How's this?: | ||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
This writes a JSON document like the one shown above to "results.json" in the current directory: https://github.com/mongodb/mongo-c-driver-performance/commit/f90e05c20 The driver benchmark spec talks about performance in terms of MB/sec of data processed, so I calculate the median value for that number for each benchmark and use that as "ops_per_sec". | ||||||||||||||||||||||||||||||||
| Comment by Brian McCarthy [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
I just tested that format out locally and I can see it in the dashboard. | ||||||||||||||||||||||||||||||||
| Comment by David Daly [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
Yes, I think that should do it. | ||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
Thanks, so my code should store one file like this?:
... then the perf visualization tools can take it from there? | ||||||||||||||||||||||||||||||||
| Comment by David Daly [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
Brian has the correct names on there. The initial support was based off of the results mongo-perf made. I believe evergreen is filling in the top level fields, and you want to create the sub document rooted in "data". The visualization is mainly using the test name, an ops_per_second value. | ||||||||||||||||||||||||||||||||
| Comment by Brian McCarthy [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
I believe the JSON file can be named whatever we like and saved in any directory. Then we specify the filename in the command in the evergreen yaml file:
For the other questions, in addition to David, we might ask mpobrien or kyle.erf | ||||||||||||||||||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 30/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
questions for brian.mccarthy or david.daly:
| ||||||||||||||||||||||||||||||||
| Comment by Brian McCarthy [ 29/Sep/16 ] | ||||||||||||||||||||||||||||||||
|
Excerpted example from https://evergreen.mongodb.com/plugin/json/task/sys_perf_linux_1_node_replSet_core_workloads_WT_7f6eb7f2361addf3f75308bf1603caa8cdc71b68_16_09_28_21_53_17/perf/
|