[SERVER-12497] role_management_helpers and user_management_helpers deadlocked with eval Created: 19/Nov/13  Updated: 03/Mar/18  Resolved: 07/Feb/14

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: None
Fix Version/s: 2.6.0-rc0

Type: Bug Priority: Critical - P2
Reporter: Eric Milkie Assignee: Andy Schwerin
Resolution: Done Votes: 0
Labels: build-failure
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-11845 parallel/basic.js timed out on OSX C+... Closed
Related
related to SERVER-33648 Attempting to perform user- and role-... Closed
Operating System: ALL
Participants:
Linked BF Score: 0

 Description   

http://buildlogs.mongodb.org/mci_0.9_osx_108/builds/10987/test/parallel_0/basicPlus.js



 Comments   
Comment by Githook User [ 07/Feb/14 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-12497 Do not call AuthorizationSession::startRequest while processing for DBDirectClient.

This eliminates a deadlock cycle between the global or admin-database write lock and the
AuthorizationManager's fetch-mode critical section.
Branch: master
https://github.com/mongodb/mongo/commit/3a7950fcf73eaf96dacddf522a8e76531157dcdd

Comment by Githook User [ 06/Feb/14 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-12497 Remove unused, questionable behaviors from ThreadSafeString.
Branch: master
https://github.com/mongodb/mongo/commit/971cd7f952be5ccfddc518347e87f53e8f4ab22a

Comment by Githook User [ 06/Feb/14 ]

Author:

{u'username': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-12497 Eliminate unused, possibly broken Client::resetThread() method.
Branch: master
https://github.com/mongodb/mongo/commit/dc70aa94f66a383cd5e9e791ec633d12055c62a7

Comment by Andy Schwerin [ 06/Feb/14 ]

I have confirmed hypothesis (1), above. The fix is to skip refreshing the session's user data cache when auth checks will be ignored, anyways (as in DBDirectClient). Code review to follow.

Comment by Andreas Nilsson [ 27/Jan/14 ]

http://codereview.10gen.com/4602662980943872/

Comment by Andy Schwerin [ 25/Nov/13 ]

Two hypotheses:

  1. The count command, executed under an "eval" expression, causes the DBDirectClient to run AuthorizationSession::startRequest() while the "eval" holds the write lock. If the eval thread tries to get the user cache mutex while holding a database or global write lock, it can deadlock with any user management command because it is acquiring its locks out of order.
  2. Maybe the currentOp command has some suspicious behavior, since evalc.js runs that command a lot.
Comment by Eric Milkie [ 25/Nov/13 ]

Interestingly, evalc.js was running when each of the above tests deadlocked.

Comment by Eric Milkie [ 25/Nov/13 ]

Another deadlock, this time with user_management_helpers:
http://buildlogs.mongodb.org/mci_0.9_osx_108_cxx11_debug/builds/11077/test/parallel_0/basic.js

Generated at Thu Feb 08 03:28:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.