[SERVER-73846] Reduce Migrations via Thread affinity on 5.0.13 Created: 09/Feb/23  Updated: 07/Mar/23  Resolved: 07/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.13
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Chris Kelly Assignee: Backlog - Service Architecture
Resolution: Incomplete Votes: 0
Labels: pull-request
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Server: Icelake server
OS: Ubuntu 20.04.2 LTS
Kernel: 5.4.0-65-generic
Compiler: gcc version 10.3.0 (Ubuntu 10.3.0-1ubuntu1~20.04)


Issue Links:
Related
is related to SERVER-74243 Evaluate the implications of pinning ... Backlog
Assigned Teams:
Service Arch
Participants:
Linked BF Score: 100

 Description   

During the testing of Mongodb with YCSB client (at the time of experiment the YCSB java code only works with v5.0 of the server), uttampc discovered a large number of migrations statistics as reported by "perf stat" command. See details below,

Event Baseline Baseline
Threads 40 80
Throughput 337,008 304,580
sched:sched_process_free 13 18
sched:sched_process_exit 13 16
sched:sched_process_wait 34 34
sched:sched_process_fork 11 15
sched:sched_process_exec 10 11
sched:sched_process_hang 0 0
task:task_newtask 11 15
task:task_rename 10 14
L1-dcache-loads 1,406,341,217,349 1,321,016,036,512
L1-dcache_load_misses 140,998,950,663 148,306,686,642
cycles 8,938,835,361,677 9,556,957,123,623
cs 199,014,407 179,466,722
migrations 411,085 37,726,014
Elapsed time (sec) 60 60

As you can see from the table above, the migrations stats is quite high. With addition of few lines of "thread affinity" code in the "runFunc" function which is a thread function per connection, I was able to see almost 70% reduction in this number with a small improvement (2% to 5%) in overall throughput.

New data snapshot,

Event Thread-affinity Thread-affinity
Threads 40 80
Throughput 342,339 325,860
migrations 38,418 9,116,093
Elapsed time (sec) 60 60

I started with upstream docker version of mongodb server which was based on v5.0.13 at the time hence this PR is based on v5.0 branch.

No. records: 40M
Type of Request: Read-Only queries.

YCSB Commands:
Loading of the database:

$ python2.7 ./bin/ycsb load mongodb -jvm-args="-Dlogback.configurationFile=../logback.xml" -s -P workloads/workloadc -threads 20 -p mongodb.url=mongodb://localhost/ycsb

Execution of queries (40 threads)

$ python2.7 ./bin/ycsb run mongodb -jvm-args="-Dlogback.configurationFile=../logback.xml" -s -P workloads/workloadc -threads 40 -p mongodb.url=mongodb://localhost/ycsb

Execution of queries (80 threads)

$ python2.7 ./bin/ycsb run mongodb -jvm-args="-Dlogback.configurationFile=../logback.xml" -s -P workloads/workloadc -threads 80 -p mongodb.url=mongodb://localhost/ycsb

Perf command

$ sudo perf stat -e "sched:sched_process*,task:*,L1-dcache-loads,L1-dcache-load-misses,cycles,cs,faults,migrations" -d -d -d -a – sleep 60

PR: https://github.com/mongodb/mongo/pull/1525 



 Comments   
Comment by NOVALUE Uttam [ 09/Feb/23 ]

Just checking the access. 

Generated at Thu Feb 08 06:25:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.