-
Type:
Bug
-
Resolution: Won't Do
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.3
-
Component/s: Sharding
-
Labels:
-
ALL
-
0
Using a single shard and single mongos shows ~17% read throughput decrease and ~20% read latency increase between 3.4.13 and 3.6.3. The attached read_only_perf_test.js
script contains a minimal repro of this regression, but it is also visible in the YCSB 100% reads benchmark.
3.4.13: 20 sec: 783760 operations; 39908.2 current ops/sec; [READ AverageLatency(us)=998.61] 30 sec: 1177532 operations; 39373.26 current ops/sec; [READ AverageLatency(us)=1012.16] 40 sec: 1577481 operations; 39994.9 current ops/sec; [READ AverageLatency(us)=996.42] 50 sec: 1979463 operations; 40194.18 current ops/sec; [READ AverageLatency(us)=991.48] 60 sec: 2372148 operations; 39268.5 current ops/sec; [READ AverageLatency(us)=1014.99] 70 sec: 2768010 operations; 39582.24 current ops/sec; [READ AverageLatency(us)=1006.98] 80 sec: 3141892 operations; 37388.2 current ops/sec; [READ AverageLatency(us)=1066.3] 3.6.3: 20 sec: 646764 operations; 33782.6 current ops/sec; [READ AverageLatency(us)=1180.32] 30 sec: 981056 operations; 33425.86 current ops/sec; [READ AverageLatency(us)=1192.9] 40 sec: 1307640 operations; 32658.4 current ops/sec; [READ AverageLatency(us)=1221.14] 50 sec: 1643276 operations; 33560.24 current ops/sec; [READ AverageLatency(us)=1188.12] 60 sec: 1977689 operations; 33441.3 current ops/sec; [READ AverageLatency(us)=1192.55] 70 sec: 2299412 operations; 32169.08 current ops/sec; [READ AverageLatency(us)=1239.63] 80 sec: 2625286 operations; 32587.4 current ops/sec; [READ AverageLatency(us)=1223.75]
If only mongos is downgraded to 3.4.13, with the shard still at 3.6.3, the performance is back to normal, so the problem must be on mongos.
Same problem is visible between 3.4.13 and master (at the time of 3.7.3):
3.4:
{
"note" : "values per second",
"errCount" : NumberLong(0),
"trapped" : "error: not implemented",
"findOneLatencyAverageMicros" : 768.6047960983534,
"totalOps" : NumberLong(1541503),
"totalOps/s" : 51348.335034725926,
"findOne" : 51348.335034725926,
"insert" : 0,
"delete" : 0,
"update" : 0,
"query" : 0,
"command" : 0
}
master:
{
"note" : "values per second",
"errCount" : NumberLong(0),
"trapped" : "error: not implemented",
"findOneLatencyAverageMicros" : 987.8736065215004,
"totalOps" : NumberLong(1192609),
"totalOps/s" : 39725.913914800716,
"findOne" : 39725.913914800716,
"insert" : 0,
"delete" : 0,
"update" : 0,
"query" : 0,
"command" : 0
}