Uploaded image for project: 'Evergreen'
  1. Evergreen
  2. EVG-12866

Add performance-data.send command to evergreen

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: current_quarter
    • Component/s: app
    • Labels:
      None

      Description

      As a DAG engineer, I'd like to add the following command to evergreen, such that, we have a simple method for transmitting performance data to expanded metrics w/o using cedar.

      User facing syntax (very similar to json.send, but with an expected schema):

      This command stores performance data for this task and notifies our performance monitoring systems of the update.

      Parameters:

      file: the JSON-formatted performance data file to save to Evergreen's DB and notify performance monitoring systems of.

      The schema for this file is the following:

      [
        {
          "test": "your_test_name_here",
          "args": {
            "thread_level": 5,
            "workload": "my_workload",
            "storageEngine": "wiredtiger",
            ...etc
          },
          "measurements": [{
            "name": "Latency50thPercentile",
            "value": 1238712371
          }, ...etc]
        }
      ]
      

      Machine workflow

      Evergreen should perform the following in order to execute this command:
      1. Validate the schema of the reported data. Fail the command if the data is not formatted correctly.
      2. Create a new document with metadata from evergreen:

        {
          "name": "performance-data",
          "task_name": "linkbench",
          "project_id": "sys-perf",
          "task_id": "sys_perf_linux_3_node_replSet_linkbench_patch_a1702ca1fccd9054111ec58bca163893a794ade1_5ad0ac28e3c33145794a675d_18_04_13_13_12_04",
          "build_id": "sys_perf_linux_3_node_replSet_patch_a1702ca1fccd9054111ec58bca163893a794ade1_5ad0ac28e3c33145794a675d_18_04_13_13_12_04",
          "variant": "linux-3-node-replSet",
          "version_id": "5ad0ac28e3c33145794a675d",
          "create_time": "2018-04-13T13:12:04.447Z",
          "is_patch": true,
          "order": 11751,
          "revision": "a1702ca1fccd9054111ec58bca163893a794ade1",
          "data": <user submitted data array>,
          "tag": ""
        }
      

      3. Store this in the same place as json.send, under the "performance-data" key/name. This should make this data available through the json history routes for our performance monitoring service to consume. Fail the command if unable to store the data.

      4. Notify signal processing service of the update:

      POST $URL/time_series/update/evergreen
      {
          task_id: "sys_perf_linux_3_node_replSet_linkbench_patch_a1702ca1fccd9054111ec58bca163893a794ade1_5ad0ac28e3c33145794a675d_18_04_13_13_12_04"
      }
      

      The credentials for making requests to SPS are already provided in Cedar. DO NOT FAIL THE COMMAND IF THIS CALL FAILS. This should generate a warning perhaps, but our service will consider the data for this run on the subsequent run. As long as evergreen has stored the data, we are good.

      Summary:
      This is more or less the same thing json.send currently does, but with less flexibility, and an extra call to performance monitoring service. We can probably reuse code from json.send and cedar to accomplish the above without much work needed.

        Attachments

        1. cedar_report.json
          5 kB
        2. perf.json
          10 kB

          Activity

            People

            Assignee:
            julian.edwards Julian Edwards
            Reporter:
            alexander.costas Alexander Costas
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: