-
Type: Task
-
Resolution: Unresolved
-
Priority: Unknown
-
None
-
Affects Version/s: None
-
Component/s: Performance Benchmarking
-
None
-
Python Drivers
Context
Ever since PYTHON-4350 we have been seeing that some of the benchmarks are widely inconsistent. For example, GridFsMultiFileDownload:
Before PYTHON-4350, GridFsMultiFileDownload ran in 12 seconds:
[2024/04/08 16:36:50.157] test/performance/perf_test.py::TestGridFsMultiFileDownload::runTest Completed TestGridFsMultiFileDownload 2194.621 MB/s, MEDIAN=0.119s, total time=12.557s
After PYTHON-4350, GridFsMultiFileDownload runs in 30.705 seconds:
[2024/04/09 19:27:45.464] test/performance/perf_test.py::TestGridFsMultiFileDownload::runTest Completed TestGridFsMultiFileDownload 2092.982 MB/s, MEDIAN=0.125s, total time=30.705s, iterations=239
It may seem counter intuitive but I think we can fix this by reducing the benchmark runtime back to 10-12 seconds by lowering MIN_ITERATION_TIME from 30 to 10. One explanation is that the mongod is suffering from a problem similar to HELP-58235.
- related to
-
PYTHON-4350 Faster and more consistent performance benchmark execution times
- Closed