-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines - Foundations
-
92.838
-
None
-
None
Currently, after completing WT-17484, we have YCSB benchmarks running in our regular CI performance testing. However, the current wtperf configuration for YCSB still has many differences compared to the MongoDB one:
- Currently, wtperf reports average read latency in microseconds, while our average read latency is usually under 1 microsecond, making this metric not very useful. We should report it in nanoseconds instead.
- MongoDB uses a scrambled Zipfian distribution, while wtperf uses a Pareto distribution (with an accent on the 0 key).
- MongoDB uses the compressibility=3 parameter, which fills values with 33% random characters and 66% "a" characters, allowing the compressed value size to be predictably around 3x smaller than the original one.
- We should not forget to increase the number of entries if we introduce compressibility, so that the total amount of disk space consumed remains similar.
- MongoDB has retryWrites, while WT always rolls back transactions on conflicts. Currently, we observe around 100k–300k update conflicts during 30-minute workloads with updates, which translates to roughly 55 events happening every second, so this may cause behavioral differences.
- MongoDB has two YCSB benchmark variants (in-cache and out-of-cache), while wtperf currently only has the in-cache variant.
- MongoDB also has YCSB load and YCSB stepdown benchmarks, although I am not sure whether they should also be ported to WT.
We should either bring the existing YCSB variants closer to the MongoDB implementation where possible, or introduce new variants whose logic matches the current MongoDB YCSB behavior as closely as possible. Those variants should be marked as HVW-matching.
- is related to
-
WT-17484 Report perf when running wtperf YCSB in CI
-
- Closed
-