[SERVER-74471] Prune existing tests to stabilize the signal from the change stream perf suite Created: 01/Mar/23  Updated: 29/Oct/23  Resolved: 16/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 7.0.0-rc1, 6.0.7, 5.0.19, 4.4.23

Type: Task Priority: Major - P3
Reporter: Sebastien Mendez Assignee: Mindaugas Malinauskas
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
is related to SERVER-76236 Extend the change_stream_listen_throu... Backlog
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.0, v6.3, v6.0, v5.0, v4.4
Sprint: QE 2023-05-15, QE 2023-05-29
Participants:
Linked BF Score: 35

 Description   
  • For the change_stream_latency tests and its derivatives, run only a single configuration: 1KB documents, ~24 worker threads, 1 collection, 1 change stream. Keep the pre-image variants, consider similarly splitting the updateLookup test into its own variant. Consider getting rid of the findOne worker since that just adds load with no direct impact on the test.
  • Consider getting rid of the change_stream_throughput tests entirely. These don't measure anything to do with change streams itself, but instead measure the impact of increasing numbers of change streams on an ongoing read/write workload. One benefit of having these tests around is that they can be correlated against the corresponding change_stream_latency tests; if write throughput improves, change streams latency regresses. But we could get the same information by simply recording write throughput in the change_stream_latency tests.
  • Consider extending the change_stream_listen_throughput test, which is currently insert-only, with mixed-workload, updateLookup and pre-image variants. This doesn't have to happen under this ticket, since it would involve adding new tests rather than reworking the current ones.

Bernard's comment



 Comments   
Comment by Githook User [ 16/May/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-74471 Prune change stream perf tests to reduce variability of the performance measurements
Branch: v4.4
https://github.com/mongodb/mongo/commit/2c6f59b45bda42dc323c5f1cab8dd36da96c1783

Comment by Githook User [ 16/May/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-74471 Prune change stream perf tests to reduce variability of the performance measurements
Branch: v5.0
https://github.com/mongodb/mongo/commit/b728afb9e22c177765e71a0f6d21d5405dbf06ff

Comment by Githook User [ 15/May/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-74471 Prune change stream perf tests to reduce variability of the performance measurements

(cherry picked from commit 837d5a5db797a3f192ff8a087725189bf445c945)
(cherry picked from commit 56b20700fb7c2e31d8a5d413f278ce4595d54b4c)
Branch: v6.0
https://github.com/mongodb/mongo/commit/cb0756c83697618126a79cb4e3a04d69ee9ea9d1

Comment by Githook User [ 02/May/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-74471 Prune change stream perf tests to reduce variability of the performance measurements

(cherry picked from commit 837d5a5db797a3f192ff8a087725189bf445c945)
Branch: v7.0
https://github.com/mongodb/mongo/commit/56b20700fb7c2e31d8a5d413f278ce4595d54b4c

Comment by Githook User [ 25/Apr/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-74471 Prune change stream perf tests to reduce variability of the performance measurements
Branch: master
https://github.com/mongodb/mongo/commit/837d5a5db797a3f192ff8a087725189bf445c945

Generated at Thu Feb 08 06:27:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.