[SERVER-13489] some query stage constructors touch disk Created: 04/Apr/14  Updated: 11/Jul/16  Resolved: 09/Apr/14

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.6.0-rc3
Fix Version/s: 2.6.1, 2.7.0

Type: Bug Priority: Major - P3
Reporter: hari.khalsa@10gen.com Assignee: hari.khalsa@10gen.com
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Backport Completed:
Participants:

 Description   

Write commands use PageFaultRetryableSection(s) and also use query. Some query constructors touch disk, and since there's a PFRS, the disk access may throw. This may cause memory leaks. Also, in the DEV case we sometimes hit an fassert. We should fix this for 2.6.1.



 Comments   
Comment by Githook User [ 10/Apr/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13489 move possible disk access out of index cursor ctor

(cherry picked from commit ae2dc0a937cd66c249d0d98802fe8229e1a32df8)
Branch: v2.6
https://github.com/mongodb/mongo/commit/a280cb5a78e94229598c3a540a933e18037dbf5c

Comment by Githook User [ 10/Apr/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13489 move disk reading out of index scan ctor

(cherry picked from commit 5f3174ac46b169a90e1e17a7327a21824609abbf)
Branch: v2.6
https://github.com/mongodb/mongo/commit/f5544e542d72c990fe4a3588ad632dd3acefe40b

Comment by Githook User [ 09/Apr/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13489 move possible disk access out of index cursor ctor
Branch: master
https://github.com/mongodb/mongo/commit/ae2dc0a937cd66c249d0d98802fe8229e1a32df8

Comment by Githook User [ 09/Apr/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13489 move disk reading out of index scan ctor
Branch: master
https://github.com/mongodb/mongo/commit/5f3174ac46b169a90e1e17a7327a21824609abbf

Comment by Eric Milkie [ 04/Apr/14 ]

All the places I found were due to IndexScan constructor calling isMultikey(). I would fix this first and then rerun tests in debug mode with likelyInPhysicalMemory()'s "mod"'s changed to 10 in all cases.

Comment by Eric Milkie [ 04/Apr/14 ]

core/auth1.js, same class but different place:

2014-04-04T15:14:02.183-0400 [conn33] found a constructor in the call tree: mongo::BtreeIndexCursor::BtreeIndexCursor(mongo::IndexCatalogEntry const*, mongo::BtreeInterface*)
_ZN5mongo16BtreeIndexCursorC2EPKNS_17IndexCatalogEntryEPNS_14BtreeInterfaceE
2014-04-04T15:14:02.183-0400 [conn33] admin.system.users Fatal Assertion 16236
2014-04-04T15:14:02.193-0400 [conn33] admin.system.users 0x1823a53 0x17b51c6 0x179b574 0xeb2582 0x159fdb3 0xfe95d9 0x159fc53 0x15c713d 0x101887e 0x1018527 0x10185e2 0x1248eb8 0x1242402 0x1190e47 0x1191411 0x118fa73 0x1461472 0x1493287 0x1310ac9 0x126a1a0 
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0x1823a53]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo10logContextEPKc+0xc6) [0x17b51c6]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo13fassertFailedEi+0x84) [0x179b574]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo7fassertEib+0x32) [0xeb2582]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x143) [0x159fdb3]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo6Record4dataEv+0x19) [0xfe95d9]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo7DiskLoc3objEv+0x33) [0x159fc53]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo16NamespaceDetails23_catalogFindIndexByNameERKNS_10StringDataEb+0x6d) [0x15c713d]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry8_indexNoEv+0x4e) [0x101887e]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry12_catalogHeadEv+0x37) [0x1018527]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry4headEv+0x32) [0x10185e2]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo16BtreeIndexCursorC2EPKNS_17IndexCatalogEntryEPNS_14BtreeInterfaceE+0xa8) [0x1248eb8]
 /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo22BtreeBasedAccessMethod9newCursorEPPNS_11IndexCursorE+0x52) [0x1242402]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo9IndexScan15initIndexCursorEv+0x87) [0x1190e47]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo9IndexScan4workEPm+0x61) [0x1191411]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo10FetchStage4workEPm+0xd3) [0x118fa73]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_7DiskLocE+0x132) [0x1461472]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo20SingleSolutionRunner7getNextEPNS_7BSONObjEPNS_7DiskLocE+0x37) [0x1493287]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo14DeleteExecutor7executeEv+0xd79) [0x1310ac9]
 /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo14receivedDeleteERNS_7MessageERNS_5CurOpE+0x500) [0x126a1a0]
2014-04-04T15:14:02.193-0400 [conn33] 

core/bulk_api_unordered.js, same trace. bulk_api_ordered.js. delx.js.

Comment by Eric Milkie [ 04/Apr/14 ]

So far, only found one place, by cranking up the lying likelyInPhysicalMemory() debug randomizer.
Running replsets/replset7.js:

 m31000| 2014-04-04T13:10:50.441-0400 [conn1] found a constructor in the call tree: mongo::IndexScan::IndexScan(mongo::IndexScanParams const&, mongo::WorkingSet*, mongo::MatchExpression const*)
 m31000| _ZN5mongo9IndexScanC2ERKNS_15IndexScanParamsEPNS_10WorkingSetEPKNS_15MatchExpressionE
 m31000| 2014-04-04T13:10:50.441-0400 [conn1] d.c Fatal Assertion 16236
 m31000| 2014-04-04T13:10:50.451-0400 [conn1] d.c 0x1823a53 0x17b51c6 0x179b574 0xeb2582 0x159fdb3 0xfe95d9 0x159fc53 0x15c713d 0x101887e 0x101858c 0x10186d4 0x1010977 0x1013082 0x1190922 0x1493dde 0x1493ea9 0x1495ccf 0x1496646 0x1417483 0x1414c57 
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x23) [0x1823a53]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo10logContextEPKc+0xc6) [0x17b51c6]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo13fassertFailedEi+0x84) [0x179b574]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo7fassertEib+0x32) [0xeb2582]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x143) [0x159fdb3]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo6Record4dataEv+0x19) [0xfe95d9]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo7DiskLoc3objEv+0x33) [0x159fc53]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo16NamespaceDetails23_catalogFindIndexByNameERKNS_10StringDataEb+0x6d) [0x15c713d]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry8_indexNoEv+0x4e) [0x101887e]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry18_catalogIsMultikeyEv+0x2c) [0x101858c]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo17IndexCatalogEntry10isMultikeyEv+0x24) [0x10186d4]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo12IndexCatalog10isMultikeyEPKNS_15IndexDescriptorE+0x77) [0x1010977]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZNK5mongo15IndexDescriptor10isMultikeyEv+0x32) [0x1013082]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo9IndexScanC2ERKNS_15IndexScanParamsEPNS_10WorkingSetEPKNS_15MatchExpressionE+0xc2) [0x1190922]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo11buildStagesERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x55e) [0x1493dde]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo11buildStagesERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x629) [0x1493ea9]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo11buildStagesERKNS_13QuerySolutionEPKNS_17QuerySolutionNodeEPNS_10WorkingSetE+0x244f) [0x1495ccf]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo12StageBuilder5buildERKNS_13QuerySolutionEPPNS_9PlanStageEPPNS_10WorkingSetE+0xa6) [0x1496646]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo19getRunnerAlwaysPlanEPNS_10CollectionEPNS_14CanonicalQueryERKNS_18QueryPlannerParamsEPPNS_6RunnerE+0xf33) [0x1417483]
 m31000|  /media/DATA2/data/m/ent/mongo/mongod(_ZN5mongo9getRunnerEPNS_10CollectionEPNS_14CanonicalQueryEPPNS_6RunnerEm+0xf47) [0x1414c57]
 m31000| 2014-04-04T13:10:50.451-0400 [conn1] 
 m31000| 

Generated at Thu Feb 08 03:31:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.