[SERVER-42695] M/R Agg: Compare performance against legacy implementation and investigate regressions Created: 07/Aug/19  Updated: 10/Jan/20  Resolved: 12/Dec/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: James Wahlin Assignee: Nicholas Zolnierz
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File mr_agg_10.svg     File mr_legacy_10.svg    
Issue Links:
Depends
depends on SERVER-42680 M/R Agg: Generate full pipeline from ... Closed
depends on SERVER-43796 Explore options for supporting multip... Closed
Related
is related to SERVER-44284 MR Agg: Avoid injecting 'emit' native... Closed
Sprint: Query 2019-11-04, Query 2019-11-18, Query 2019-12-02, Query 2019-12-16
Participants:
Linked BF Score: 0

 Comments   
Comment by James Wahlin [ 12/Dec/19 ]

Worth noting is that the above results all represent a single-threaded client performance workload. It turns out that our mapReduce sys-perf tests would need to be modified to support a multiple client thread test. The impact of this is that we will not see the gains we expect from no longer taking a global write lock when M/R writes to an output collection.

Comment by Nicholas Zolnierz [ 12/Dec/19 ]

Final sys-perf patch build is here, which includes that commit that removed the feature flag as well as SERVER-44334 from above. Note that the base commit also includes the feature flag removal, so its best to compare against an older commit (e.g. 1f5591c7a060cc9a048462b618e2ca7c767036c3 for standalone). To summarize the results:

Standalone

Test Max max @1f5591c vs 1f5591c
map_reduce_doc_inline 68,447 93,249 -26.60%
map_reduce_doc_reduce 65,441 86,210 -24.09%
map_reduce_doc_merge 65,930 86,378 -23.67%
map_reduce_doc_replace 65,067 85,473 -23.87%
word_count_doc 17,589 30,685 -42.68%

 Shard-lite cluster (bestbuy only)

Test Max max @7290512 vs 7290512
filtered_word_count_map_reduce 5,267 3,334 57.98%
all_word_count_map_reduce 2,622 1,200 118.48%

 3-shard cluster

Test Max max @64a9f0d vs 64a9f0d
map_reduce_doc_inline 145,378 178,192 -18.42%
map_reduce_doc_reduce 143,321 179,293 -20.06%
map_reduce_doc_merge 142,867 178,479 -19.95%
map_reduce_doc_replace 141,511 180,987 -21.81%
word_count_doc 45,994 43,763 5.10%

 AFAIK, these results include any relevant perf fixes from the "Unity projection implementation" project as well as any perf improvements from the M/R in agg project. Mostly posting the numbers for posterity as we don't plan to spend any additional time on this as part of PM-766. 

CC asya

 

Generated at Thu Feb 08 05:01:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.