[SERVER-10181] fassert during parallel/basicPlus.js on Linux 64-bit DEBUG Created: 11/Jul/13  Updated: 11/Jul/16  Resolved: 24/Jul/13

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 2.5.2

Type: Bug Priority: Major - P3
Reporter: Matt Kangas Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: buildbot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Linux 64-bit DEBUG Build #2320


Issue Links:
Related
related to SERVER-9778 $where causes Matcher2 to access a da... Closed
Operating System: ALL
Participants:

 Description   

Linux 64-bit DEBUG Build #2320

Given that this is a parallel test, I don't know which test actually triggered this failure.

http://buildbot.mongodb.org/builders/Linux%2064-bit%20DEBUG/builds/2320/steps/test_14/logs/stdio
http://buildlogs.mongodb.org/Linux%2064-bit%20DEBUG/builds/2320/test/parallel/basicPlus.js

Wed Jul 10 17:32:07.615 [conn81] test.system.js Fatal Assertion 16236
Wed Jul 10 17:32:07.637 [conn81] test.system.js 0x11d8441 0x11b6c25 0x11a27e6 0xbe8d74 0x1009bb0 0xd99cee 0xd99f08 0xd98562 0xf30dfc 0xf33397 0xeb6209 0xeb721c 0xebabe3 0xc70905 0xc467c1 0xebadce 0x116b3d3 0x116d5e4 0x116c260 0xef9596 
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x29) [0x11d8441]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo10logContextEPKc+0x94) [0x11b6c25]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo13fassertFailedEi+0x62) [0x11a27e6]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo7fassertEib+0x29) [0xbe8d74]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x11c) [0x1009bb0]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo6Record7getNextERKNS_7DiskLocE+0x24) [0xd99cee]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZNK5mongo7Forward4nextERKNS_7DiskLocE+0x34) [0xd99f08]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo11BasicCursor7advanceEv+0x102) [0xd98562]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_12ChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE+0x701) [0xf30dfc]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x129b) [0xf33397]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod() [0xeb6209]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x385) [0xeb721c]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x91) [0xebabe3]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBClientCursor4initEv+0xd7) [0xc70905]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xe9) [0xc467c1]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x6e) [0xebadce]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo5Scope10loadStoredEb+0x193) [0x116b3d3]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo11PooledScope10loadStoredEb+0x3e) [0x116d5e4]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo12ScriptEngine14getPooledScopeERKSsS2_+0x206) [0x116c260]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo20WhereMatchExpression4initERKNS_10StringDataES3_RKNS_7BSONObjE+0x286) [0xef9596]
Wed Jul 10 17:32:07.637 [conn81] 
***aborting after fassert() failure
 
********** Parallel Test FAILED: Error: error loading js file: jstests/geo_s2nearComplex.js
********** Parallel Test FAILED: Error: error loading js file: jstests/sortm.js
********** Parallel Test FAILED: Error: error loading js file: jstests/use_power_of_2.js
********** Parallel Test FAILED: Error: error loading js file: jstests/constructors.js
assert: [0] != [4] are not equal : one or more tests failed



 Comments   
Comment by auto [ 24/Jul/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-10181 Stop looking up stack when hitting an exempt constructor
Branch: master
https://github.com/mongodb/mongo/commit/3a4fe89b3447d71195a2f70be235b00223fc4288

Comment by Matt Kangas [ 17/Jul/13 ]

Another example of this failure? This time under parallel/basic.js, but subtest is again constructors.js.

Linux 64-bit DEBUG Build #2333 July 16 (rev 2cf30807f8)

http://buildbot.mongodb.org/builders/Linux%2064-bit%20DEBUG/builds/2333/steps/test_14/logs/stdio
http://buildlogs.mongodb.org/Linux%2064-bit%20DEBUG/builds/2333/test/parallel/basic.js

Wed Jul 17 00:21:49.151 [conn29] query test.where_constructors query: { $where: "Timestamp()" } ntoreturn:1 ntoskip:0 nscanned:1 keyUpdates:0 locks(micros) r:254335 nreturned:1 reslen:53 254ms
 
Wed Jul 17 00:21:49.276 [conn29] Error: Timestamp needs 0 or 2 arguments
Wed Jul 17 00:21:49.278 [conn29] assertion 16722 Error: Timestamp needs 0 or 2 arguments ns:test.where_constructors query:{ $where: "Timestamp(0)" }
Wed Jul 17 00:21:49.278 [conn29]  ntoskip:0 ntoreturn:-1
found a constructor in the call tree: mongo::CoveredIndexMatcher::CoveredIndexMatcher(mongo::BSONObj const&, mongo::BSONObj const&)
_ZN5mongo19CoveredIndexMatcherC1ERKNS_7BSONObjES3_
Wed Jul 17 00:21:49.571 [conn29] test.system.js Fatal Assertion 16236
Wed Jul 17 00:21:49.593 [conn29] test.system.js 0x1200ced 0x11df52d 0x11cb0ee 0xbfbd70 0x1032e1c 0xdadbbe 0xdaddd8 0xdac432 0xf44c18 0xf471b3 0xeca0bd 0xecb0d0 0xece9b1 0xc8352d 0xc5940d 0xeceb9c 0x11937f7 0x1195a08 0x1194684 0xf0d362 
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x29) [0x1200ced]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo10logContextEPKc+0x94) [0x11df52d]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo13fassertFailedEi+0x62) [0x11cb0ee]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo7fassertEib+0x29) [0xbfbd70]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x11c) [0x1032e1c]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo6Record7getNextERKNS_7DiskLocE+0x24) [0xdadbbe]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZNK5mongo7Forward4nextERKNS_7DiskLocE+0x34) [0xdaddd8]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo11BasicCursor7advanceEv+0x102) [0xdac432]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_12ChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE+0x701) [0xf44c18]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x129b) [0xf471b3]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod() [0xeca0bd]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x385) [0xecb0d0]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x91) [0xece9b1]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBClientCursor4initEv+0xd7) [0xc8352d]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xe9) [0xc5940d]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x6e) [0xeceb9c]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo5Scope10loadStoredEb+0x193) [0x11937f7]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo11PooledScope10loadStoredEb+0x3e) [0x1195a08]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo12ScriptEngine14getPooledScopeERKSsS2_+0x206) [0x1194684]
 /data/buildslaves/Linux_64bit_DEBUG/mongo/mongod(_ZN5mongo20WhereMatchExpression4initERKNS_10StringDataES3_RKNS_7BSONObjE+0x286) [0xf0d362]
Wed Jul 17 00:21:49.593 [conn29] 
***aborting after fassert() failure
Wed Jul 17 00:21:49.716 DBClientCursor::init call() failed
Wed Jul 17 00:21:49.718 Error: error doing query: failed at src/mongo/shell/query.js:81
********** Parallel Test FAILED: Error: error loading js file: jstests/counta.js
Wed Jul 17 00:21:49.719 Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999
Wed Jul 17 00:21:49.719 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 
Wed Jul 17 00:21:49.719 DBClientCursor::init call() failed
Wed Jul 17 00:21:49.722 Socket recv() errno:104 Connection reset by peer 127.0.0.1:27999
Wed Jul 17 00:21:49.722 SocketException: remote: 127.0.0.1:27999 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27999] 
Wed Jul 17 00:21:49.722 DBClientCursor::init call() failed
Wed Jul 17 00:21:49.724 Error: error doing query: failed at src/mongo/shell/query.js:81
Wed Jul 17 00:21:49.725 DBClientCursor::init call() failed
Wed Jul 17 00:21:49.727 trying reconnect to 127.0.0.1:27999
Wed Jul 17 00:21:49.729 reconnect 127.0.0.1:27999 failed couldn't connect to server 127.0.0.1:27999
********** Parallel Test FAILED: Error: error loading js file: jstests/capped7.js
Wed Jul 17 00:21:49.740 Error: error doing query: failed at src/mongo/shell/collection.js:54
********** Parallel Test FAILED: Error: error loading js file: jstests/orm.js
Wed Jul 17 00:21:49.761 Error: socket exception [FAILED_STATE] for 127.0.0.1:27999 at src/mongo/shell/collection.js:54
********** Parallel Test FAILED: Error: error loading js file: jstests/constructors.js
assert: [0] != [4] are not equal : one or more tests failed

Comment by Eliot Horowitz (Inactive) [ 15/Jul/13 ]

Stopping when he hit something exempt is probably correct.

Comment by Mathias Stearn [ 11/Jul/13 ]

This is caused by SERVER-9778. We are constructing a CoveredIndexMater which has a Matcher (typedefed to Matcher2) member. While we have an excemption for Matcher2 we keep looking up the stack. Instead we should probably stop once we hit an exempted Class.

In addition we are passing through a DBDirectClient which would prevent a PageFaultException from propagating anyway. We should probably stop looking up the stack once we get to a DBDirectClient boundry since at that point we know we are safe.

Comment by Matt Kangas [ 11/Jul/13 ]

Note that this builder is now clear as of #2323

http://buildbot.mongodb.org/builders/Linux%2064-bit%20DEBUG/builds/2323

Generated at Thu Feb 08 03:22:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.