[SERVER-5485] implement thread safety for shell utils, using fine grained locking Created: 03/Apr/12  Updated: 11/Jul/16  Resolved: 08/Jun/12

Status: Closed
Project: Core Server
Component/s: JavaScript, Shell
Affects Version/s: None
Fix Version/s: 2.1.2

Type: Bug Priority: Major - P3
Reporter: Ian Whalen (Inactive) Assignee: Tad Marshall
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 64-bit v8, need to see if Windows V8 has issues


Issue Links:
Related
related to SERVER-6042 Check for remaining Windows issues in... Closed
is related to SERVER-5559 shell doesn't prompt to kill backgrou... Closed
is related to SERVER-5574 db not defined error messages appear ... Closed
Operating System: ALL
Participants:

 Description   

shell: started program /data/slave/Linux_64bit_v8/mongo/mongo --eval sleep(1000); db.sps.insert({x:1}); db.getLastError(); 127.0.0.1:27999
Tue Apr  3 08:48:32 mongo got signal 11 (Segmentation fault), stack trace: 
 
sh16333| connecting to: 127.0.0.1:27999/test
sh16333| null
Tue Apr  3 08:48:32 0x47c78b 0x7f97ffc4d980 0x7f98004ae06f 0x4907a1 0x490885 0x49f850 0x499dcd 0x55bbbf 0x552306 0x5da021 0x30f4dc306302 
 /data/slave/Linux_64bit_v8/mongo/mongo(_Z12quitAbruptlyi+0x33b) [0x47c78b]
 /lib64/libc.so.6(+0x32980) [0x7f97ffc4d980]
 /usr/lib64/libstdc++.so.6(_ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0_RS_+0x5f) [0x7f98004ae06f]
 /data/slave/Linux_64bit_v8/mongo/mongo() [0x4907a1]
 /data/slave/Linux_64bit_v8/mongo/mongo() [0x490885]
 /data/slave/Linux_64bit_v8/mongo/mongo(_ZN5mongo10shellUtils13ProgramRunner5startEv+0x380) [0x49f850]
 /data/slave/Linux_64bit_v8/mongo/mongo(_ZN5mongo10shellUtils17StartMongoProgramERKNS_7BSONObjEPv+0x3d) [0x499dcd]
 /data/slave/Linux_64bit_v8/mongo/mongo(_ZN5mongo7V8Scope14nativeCallbackEPS0_RKN2v89ArgumentsE+0x5cf) [0x55bbbf]
 /data/slave/Linux_64bit_v8/mongo/mongo(_ZN5mongo7V8Scope10v8CallbackERKN2v89ArgumentsE+0x76) [0x552306]
 /data/slave/Linux_64bit_v8/mongo/mongo() [0x5da021]
 [0x30f4dc306302]

http://buildbot.mongodb.org/builders/Linux%2064-bit%20v8/builds/3256/steps/test_11/logs/stdio



 Comments   
Comment by Tad Marshall [ 08/Jun/12 ]

Created SERVER-6042 to track any remaining Windows issues. Resolving this as fixed.

Comment by Tad Marshall [ 08/Jun/12 ]

I spent some time looking at the code and there was nothing obvious that needed to be done beyond what Aaron has already done. Then I started to try to reproduce the original problem and realized that this only affects the V8 build. Since we're not supporting V8 for version 2.2, the follow-up on this ticket can be postponed until V8 moves up in the queue. Postponing to 2.3 desired for now.

Comment by auto [ 11/Apr/12 ]

Author:

{u'login': u'tadmarshall', u'name': u'Tad Marshall', u'email': u'tad@10gen.com'}

Message: SERVER-5485 Update Visual Studio for new filenames
Branch: master
https://github.com/mongodb/mongo/commit/4f52acec85df522497f874f4661a3d022a25b624

Comment by auto [ 11/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5485 Update old comments about thread safety.
Branch: master
https://github.com/mongodb/mongo/commit/e909c2f3aa58b7c986546e7752a7d688397dcba0

Comment by auto [ 11/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5485 Make 'cd' object non static.
Branch: master
https://github.com/mongodb/mongo/commit/7d23c581e858b6888f2261311b09efbcf9a2530a

Comment by auto [ 11/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5485 Reduce usage of async signal unsafe routines when forking a new process in shell utils, comment remaining async unsafe routines.
Branch: master
https://github.com/mongodb/mongo/commit/d4c4efde1b009552cde987460483f98d31292428

Comment by auto [ 11/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5485 Add locking to ProgramRegistry.
Branch: master
https://github.com/mongodb/mongo/commit/6c1cd0407427fb455f550ab77976d57683aa756f

Comment by Eliot Horowitz (Inactive) [ 05/Apr/12 ]

Yes - definitely needs to be finer grained.

Comment by Aaron Staple [ 05/Apr/12 ]

I think previously we had a pretty broad lock (which has been removed from the code) that we yielded for things like network io and sleeping. But if we want to move to more fine grained locking I can implement that.

Comment by Eliot Horowitz (Inactive) [ 04/Apr/12 ]

aaron - with multi threaded v8 the shell helpers need to be thread safe.

in this case, the process management code you wrote a while back is just completely not thread safe.

definitely don't want a big lock around it.

Comment by Antoine Girbal [ 04/Apr/12 ]

most native functions called from JS should already be thread safe.
Adding single mutex around call could have bad performance implication, like if a sleep() takes the lock.
Will review the native functions and see what is not thread safe, should be quick.

Comment by Aaron Staple [ 03/Apr/12 ]

@antoine - Is this related to removal of the js callback mutex, and do we have a general plan for dealing with issues like this? (Would it be acceptable, provided it's correct, to use a single mutex to lock all the shell utils functions for example?)

@andy - If you know the answer to this, could you confirm on mutex granularity.

Comment by Andy Schwerin [ 03/Apr/12 ]

Looks like a classic "need a mutex to guard a shared object" bug around "dbs" in shell_utils.cpp. Pretty much any native function callable from javascript needs to be threadsafe.

Generated at Thu Feb 08 03:09:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.