[SERVER-19911] Set wiredTigerConcurrentReadTransactions based on machine specs? Created: 12/Aug/15  Updated: 13/Aug/15  Resolved: 13/Aug/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Ben McCann Assignee: Michael Cahill (Inactive)
Resolution: Done Votes: 0
Labels: concurrency, docs
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File 1hour.png     PNG File 4hours.png    
Participants:

 Description   

I see that there's a setting for wiredTigerConcurrentReadTransactions. We appear to be sometimes hitting this. I'm not really sure why this exists, but it appears to potentially be artificially capping throughput? If you have a really beefy machine, then you should be able to handle more concurrent read transactions? Is there any reason we shouldn't raise it? Does it increase contention to a point where things slow down more than they are sped up?

I see a brief reference to being able to change it in the docs (http://docs.mongodb.org/manual/reference/parameters/), but it's not clear to me if I should or what the implications are.



 Comments   
Comment by Ramon Fernandez Marina [ 13/Aug/15 ]

Thanks for the detailed update chengas123. As you've found out, a more detailed analysis of specific loads is generally needed to determine which parameter to adjust and how. I am going to close this ticket for now; as we gather more information from field deployments and real-life workloads we'll certainly be considering adjustments to the default values for some of the tunables.

Regards,
Ramón.

Comment by Ben McCann [ 13/Aug/15 ]

For what it's worth, we just figured out that under normal circumstances we have almost all of our read tickets available. The reason they were being used is because we were cache constrained so reads were slowing down which made us have many reads going concurrently. Read tickets were a symptom and not root cause. We increased the cache size and immediately our read tickets used fell to 0. db.adminCommand(

{ "setParameter": 1, "wiredTigerEngineRuntimeConfig": "cache_size=110G"}

)

Comment by Ben McCann [ 12/Aug/15 ]

Thanks! To the extent that it's helpful to get details about what we're seeing, I'm happy to help. E.g. we have graphs of every stat that's in db.serverStatus() including the wiredTiger stuff that you wouldn't normally get from MMS (I've attached a couple as examples, but we have hundreds of stats. Note in the example that there was a machine reboot when everything settles down).

Comment by Ramon Fernandez Marina [ 12/Aug/15 ]

chengas123, I've moved this ticket to the SERVER project, which is where we keep track of bugs and feature requests for MongoDB (the WT project is for users of stand-alone WiredTiger).

There may not be a good rule of thumb to set this value depending on machine specs, as other things like application load should probably be taken into consideration, but we'll discuss internally – stay tuned.

Generated at Thu Feb 08 03:52:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.