[SERVER-20167] Avoid spinlock in TimerStats Created: 27/Aug/15  Updated: 19/Sep/15  Resolved: 27/Aug/15

Status: Closed
Project: Core Server
Component/s: Concurrency
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Martin Bligh
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-20091 Poor query throughput and erratic beh... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: QuInt 8 08/28/15
Participants:

 Description   

As we saw in SERVER-20091, using a spinlock in situations where a large number of threads can contend for a resource can lead to scalability issues and erratic behavior. Code inspection reveals an additional place where this might occur: TimerStats uses a spinlock to make updates to the count and the total time atomic, and this is used by gleWtimeStats, which can potentially be updated by a large number of threads at high connection count. We could avoid this by

  • using a mutex instead of a spinlock
  • don't use any lock, use atomics for the individual stats (count and total milliseconds), accept the possibility of a small skew between the two. I think this is probably acceptable for the typical uses of this statistic.

I haven't been able to demonstrate a performance issue for this because this stat is only update for write concern with w>1 or j:true, and both of those have other performance issues that dominate at the moment. Independent of that however, it seems like a good idea to forestall performance pitfalls for future uses of this class.



 Comments   
Comment by Githook User [ 31/Aug/15 ]

Author:

{u'username': u'martinbligh', u'name': u'Martin Bligh', u'email': u'mbligh@mongodb.com'}

Message: SERVER-20167: Flip stats counters to be signed (forgot const type)
Branch: v3.0
https://github.com/mongodb/mongo/commit/284e8b79c02ec00380a667c19b662fccaf61b7f4

Comment by Githook User [ 31/Aug/15 ]

Author:

{u'username': u'martinbligh', u'name': u'Martin Bligh', u'email': u'mbligh@mongodb.com'}

Message: SERVER-20167: Flip stats counters to be signed (forgot const type)
Branch: master
https://github.com/mongodb/mongo/commit/518118bfce4da06e6c26063e125f90e30a1b3baa

Comment by Githook User [ 31/Aug/15 ]

Author:

{u'username': u'martinbligh', u'name': u'Martin Bligh', u'email': u'mbligh@mongodb.com'}

Message: SERVER-20167: Flip stats counters to be signed
Branch: v3.0
https://github.com/mongodb/mongo/commit/dea495557e21478e9783bbce7c5d6b6463358052

Comment by Githook User [ 31/Aug/15 ]

Author:

{u'username': u'martinbligh', u'name': u'Martin Bligh', u'email': u'mbligh@mongodb.com'}

Message: SERVER-20167: Flip stats counters to be signed
Branch: master
https://github.com/mongodb/mongo/commit/2b209d7b214bee35b8890f33534254c7a94182f2

Comment by Martin Bligh [ 28/Aug/15 ]

Don't think so, this one is speculative from code inspection. I'll do the network stats one though

Comment by Githook User [ 27/Aug/15 ]

Author:

{u'username': u'martinbligh', u'name': u'Martin Bligh', u'email': u'mbligh@mongodb.com'}

Message: SERVER-20167: use atomics for TimerStats
Branch: master
https://github.com/mongodb/mongo/commit/c9d635404d514336a5f6ca50bf0a13f385e92bfc

Generated at Thu Feb 08 03:53:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.