Standardize performance benchmark reporting metrics

XMLWordPrintableJSON

    • Needed

      The Benchmarking spec says:

      In addition to timing data, all micro-benchmark tasks will be measured in terms of "megabytes/second" (MB/s) of documents processed, with higher scores being better. (In this document, "megabyte" refers to the SI decimal unit, i.e. 1,000,000 bytes.) This makes cross-benchmark comparisons easier.

      However, each driver's benchmark suite reports different measurements right now. Python reports "bytes_per_second", Java reports ops_per_second, C says "ops_per_second" but actually reports bytes/sec, etc..

      We should also standardize the "Test" names that are reported so that we can more easier compare results across drivers. For example, the "GridFS download" test is called "GridFS Download" in Java, "TestGridFsDownload" in Python and C, "gridFsDownload" in Node.

      Related to DRIVERS-2666.

              Assignee:
              Unassigned
              Reporter:
              Shane Harvey
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: