[SERVER-13033] Static Initalizer race condition with CreateDirectClient and server-side Javascript Created: 04/Mar/14  Updated: 11/Jul/16  Resolved: 20/Mar/14

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: 2.4.9
Fix Version/s: 2.6.0-rc2

Type: Bug Priority: Critical - P2
Reporter: Prasanna V. Loganathar Assignee: Mark Benvenuto
Resolution: Done Votes: 0
Labels: crash
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File Log.txt     File mongo.dmp    
Issue Links:
Duplicate
is duplicated by SERVER-9774 Random access exception accessing vir... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Same steps, crash the server again.
Over 500 threads, running on the same system for a few mins, and the same illegal query crashes the server.

Participants:
Linked BF Score: 0

 Description   

Server crash, on an Illegal query under heavy load.



 Comments   
Comment by Githook User [ 19/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-13033: Static Initalizer race condition with CreateDirectClient and server-side Javascript

(cherry picked from commit 7be64ae7568306dcbf75335a5e80793f4ce1268d)
Branch: v2.6
https://github.com/mongodb/mongo/commit/f5623ce7ed697e734402a75c70950ee7308b6280

Comment by Githook User [ 19/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-13033: Static Initalizer race condition with CreateDirectClient and server-side Javascript
Branch: master
https://github.com/mongodb/mongo/commit/7be64ae7568306dcbf75335a5e80793f4ce1268d

Comment by Githook User [ 18/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: Revert "SERVER-13033: Fix bad static initializer"

This reverts commit f0b367f3e04408f66fcc8f5bac1425155deec5c3.

(cherry picked from commit 1aad465958f32bad33172b03b26012a786d80a86)
Branch: v2.6
https://github.com/mongodb/mongo/commit/d1e423807afe060499c44c44863a59c764a4bc9d

Comment by Githook User [ 18/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: Revert "SERVER-13033: Fix bad static initializer"

This reverts commit f0b367f3e04408f66fcc8f5bac1425155deec5c3.
Branch: master
https://github.com/mongodb/mongo/commit/1aad465958f32bad33172b03b26012a786d80a86

Comment by Githook User [ 18/Mar/14 ]

Author:

{u'name': u'Dan Pasette', u'email': u'dan@10mongodb.com'}

Message: Revert "SERVER-13033: Fix bad static initializer"

This reverts commit c991426e46e416dfad3e433995a1bd2725e9b09c.
Branch: v2.4
https://github.com/mongodb/mongo/commit/be2d871df615e8322c5cb377426721e171830ed4

Comment by Githook User [ 17/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-13033: Fix bad static initializer
(cherry picked from commit f0b367f3e04408f66fcc8f5bac1425155deec5c3)
Branch: v2.4
https://github.com/mongodb/mongo/commit/c991426e46e416dfad3e433995a1bd2725e9b09c

Comment by Githook User [ 17/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-13033: Fix bad static initializer

(cherry picked from commit f0b367f3e04408f66fcc8f5bac1425155deec5c3)
Branch: v2.6
https://github.com/mongodb/mongo/commit/77fb13bbcec0335469af0a2fbdeb0d7dd7f687c7

Comment by Githook User [ 17/Mar/14 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-13033: Fix bad static initializer
Branch: master
https://github.com/mongodb/mongo/commit/f0b367f3e04408f66fcc8f5bac1425155deec5c3

Comment by Mark Benvenuto [ 05/Mar/14 ]

A few things
1. 2.4.9 dump is partially corrupt due to a know bug so I could not get a stack dump for the crash in question
2. We hit 3 dumps in almost parallel at the same piece of code so #1 is largely irrelevant except for the lack of variables
3. Here the code in question.

        static DBClientBase* db = createDirectClient();
        auto_ptr<DBClientCursor> c = db->query(coll, Query(), 0, 0, NULL, QueryOption_SlaveOk, 0);

The crash is occuring on the call db->query after it was loaded from the function local static variable db. I do not know why it is zero in this case.

I can think of two possibilities:
1. new threw and that left db zero
2. DBDirectClient constructor threw

The query that was run in pymongo:

.find(

{ '$where' : Code("_id = 531643994f2f680020b5f57a") }

)

Wed Mar 05 03:10:41.560 [conn402] SyntaxError: Unexpected token ILLEGAL
Wed Mar 05 03:10:41.611 [conn402] assertion 16722 SyntaxError: Unexpected token ILLEGAL ns:TestDb.Users query:{ $where: _id = 531643994f2f680020b5f57a }
Wed Mar 05 03:10:41.611 [conn402]  ntoskip:0 ntoreturn:-1
Wed Mar 05 03:10:41.611 [conn402] query TestDb.Users query: { $where: _id = 531643994f2f680020b5f57a } ntoreturn:1 keyUpdates:0 exception: SyntaxError: Unexpected token ILLEGAL code:16722 locks(micros) r:368289 reslen:83 376ms

Generated at Thu Feb 08 03:30:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.