[SERVER-6123] aggregation holds read lock indefinitely without yielding after initial processing of first sort or group stage Created: 19/Jun/12  Updated: 11/Jul/16  Resolved: 04/Jul/12

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 2.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Aaron Staple
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-6296 improve aggregation framework concurr... Closed
is related to SERVER-6286 deadlock between db mutex and sm mutex Closed
is related to SERVER-5089 aggregation needs to yield Closed
Operating System: ALL
Participants:

 Description   

The aggregate command takes a read lock for the duration of its operation. This lock may be yielded when the source Cursor is advanced (and is only yielded in this case I believe).

A sort or group stage in the pipeline will read all the results of its own source before generating its own first result. This means that yielding will not occur after the first sort or group stage of the pipeline returns its first result.

Test

c = db.c;
c.drop();
 
// Insert a million docs.
for( i = 0; i < 1*1000*1000; ++i ) {
    c.save( { a:i, b:i, c:i } );
}
db.getLastError();
print( "saved" );
 
// Run a pipeline with a large sort and some time consuming steps after it.                       
printjson( c.aggregate( { $sort:{ a:1 } }, { $project:{ d:{ $add:[ '', '$a', '.', { $multiply:[ '$a', '$b' ] }, '.', { $multiply:[ '$a', '$b', '$c' ] } ] } } }, { $limit:10 } ) );

You can observe a long lived read lock by attempting periodic writes from a separate client.

Observed behavior: An aggregate command may hold a read lock for a prolonged interval without yielding.
Expected behavior: An aggregate command should only hold a read lock when necessary to read documents or access other shared state.



 Comments   
Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T14:27:47-07:00', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-6123 Implement 'default locked' concurrency plan for aggregation.
Branch: master
https://github.com/mongodb/mongo/commit/1407bb116c839f88bf908595e4521fb373689e9c

Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T13:41:41-07:00', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-6123 Clean some aggregation related #includes.
Branch: master
https://github.com/mongodb/mongo/commit/057121832ecbf7bed6b1d83a4a3fa38b6d16143d

Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T13:40:00-07:00', u'email': u'aaron@10gen.com', u'name': u'Aaron'}

Message: SERVER-6123 Clean up Cursor usage in DocumentSourceCursor.
Branch: master
https://github.com/mongodb/mongo/commit/5c0b3ba4ac9e940747cbc9feb51c9cd14a3c5c28

Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T13:39:18-07:00', u'email': u'aaron@10gen.com', u'name': u'Aaron'}

Message: SERVER-6123 Remove unused DocumentSourceCursor::bsonDependencies member.
Branch: master
https://github.com/mongodb/mongo/commit/01b6a30140920820ac0e7d9445de90cec335987e

Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T13:38:39-07:00', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-6123 Remove unnecessary (default) arguments to ReadContext() in PipelineCommand.
Branch: master
https://github.com/mongodb/mongo/commit/3eb6d841ebd52e5b7ba5ea2b9b437ae8ae2da193

Comment by auto [ 03/Jul/12 ]

Author:

{u'date': u'2012-07-03T13:37:51-07:00', u'email': u'aaron@10gen.com', u'name': u'Aaron'}

Message: SERVER-6123 Remove unused lockGlobally() implementation in PipelineCommand.
Branch: master
https://github.com/mongodb/mongo/commit/bdc8d8617f389bbb6825046c0499396c2bc69a3a

Generated at Thu Feb 08 03:10:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.