Run benchmarks_query on amazon2023.3-arm64-large

XMLWordPrintableJSON

    • Type: Improvement
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Execution
    • None
    • 3
    • TBD
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Currently, we run benchmarks_query on rhel8.8-large. We should look at running them on amazon2023.3-arm64-large instead due to lower noise. Note: this only applies to the instruction count benchmarks, I'm not sure of the impact on the query benchmarks measuring latency (although they potentially could also benefit).

      In PERF-7067, we saw that IdPointQuery from point_query_bm had a coefficient of variation (CV) of 0.013% when running on amazon2023.3-arm64-large. This is significantly lower than the CV of 0.16% we see for the same benchmark running on rhel8.8-large on the waterfall. See this comment for more details.

      We should double check that the variant change has the same positive effect on noise on all the other instruction count queries in benchmarks_query before making this change.

      Reducing the noise on our instruction-level microbenchmarks has the potential to greatly improve our ability to detect small compounding changes in our critical code path.

              Assignee:
              Unassigned
              Reporter:
              Alice Doherty
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: