[SERVER-4892] Running server in auth mode fails to close cursors leading to cursor accumulation on the server Created: 07/Feb/12  Updated: 11/Jul/16  Resolved: 21/Feb/12

Status: Closed
Project: Core Server
Component/s: Querying, Security
Affects Version/s: 2.0.2, 2.0.3
Fix Version/s: 2.1.1

Type: Bug Priority: Critical - P2
Reporter: Christian Amor Kvalheim Assignee: Andy Schwerin
Resolution: Done Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

osx or linux, server must be set up on a different ip than localhost (preferably a separate machine). This is not a ruby driver issue, I used ruby to ensure it was not a node.js driver issue.

https://github.com/christkv/node-mongodb-native/issues/497


Attachments: Text File ruby_server.log     Text File server.log    
Issue Links:
Duplicate
is duplicated by SERVER-4600 No auth for killCursors command Closed
is duplicated by SERVER-626 killcursors should do some authentica... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

Setup for reproducing the issue

---------------------------------------- app to use

require 'sinatra'
require 'mongo'
require 'json'
 
@@conn = Mongo::Connection.new('192.168.0.192', 27017);
@@db1 = @@conn.db('tweet')
@@db1.authenticate('admin', 'admin')
 
get '/' do
  puts @@db1
 
  @coll = @@db1.collection('tweets')
  @cursor = @coll.find({}, :fields => ['_id']).limit(2)
  @results = @cursor.to_a();
  JSON.dump(@results)
end

------------------------------------------ Data population script from console

use tweet
db.tweets.drop();
 
for(var i=0;i<200;i++){
  print(i)
  db.tweets.insert({account_id:new ObjectId(), avg_ctr:Math.random(), published:new Date() })
}

------------------------------------------ Steps to reproduce
1. Boot up mongod in auth mode
2. Run the console script to set up a database with some test data
3. Set up the user for the 'tweet' db using

use tweet
db.addUser('admin', 'admin')

4. start the application

ruby server.rb (or where you stored the file)

5. Hit the url with your browser or curl a couple of times

http://localhost:4567

6. Go to the mongo console and do

db.serverStatus()['cursors']

You will see the number of open cursor increase and hang around until they time out. This does not seem to happen if you run against the server on localhost or without auth.



 Comments   
Comment by auto [ 22/Feb/12 ]

Author:

{u'login': u'tadmarshall', u'name': u'Tad Marshall', u'email': u'tad@10gen.com'}

Message: SERVER-4892 fix CRLF line endings
Branch: master
https://github.com/mongodb/mongo/commit/a5bd61d7753856918fbf2be1e1f1e14732831363

Comment by auto [ 22/Feb/12 ]

Author:

{u'login': u'', u'name': u'unknown', u'email': u'Administrator@tad-w2k8r2-vbox.(none)'}

Message: SERVER-4892 fix test so Windows version doesn't assert

Fix from Andy ... remove assert around stopMongod( port ) at
end of test in jstests/auth/server-4892.js .
Branch: master
https://github.com/mongodb/mongo/commit/661b26c5f68c6987b70c8059d03f14397f7965a0

Comment by Andy Schwerin [ 21/Feb/12 ]

Was closed by accident. Reopening marked it "unresolved", but the bug is fixed, per earlier comment.

Comment by Andy Schwerin [ 21/Feb/12 ]

Closed by accident.

Comment by Andy Schwerin [ 21/Feb/12 ]

This affects 2.0.0 and 2.0.1, as well. It probably also affects all prior versions that supported auth mode, though I haven't audited them.

As a reminder, this bug only causes cursor leaks when a query results in a large number of documents being returned or the query has explicitly set a low batchSize, and the client doesn't examine any documents from the last batch returned.

A decision on backporting this fix to 2.0.x is pending, but it's probably too late for 2.0.3.

Comment by Fredrik Björk [ 21/Feb/12 ]

Does this affect version 2.0.1 too?

Comment by auto [ 21/Feb/12 ]

Author:

{u'login': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: Regression test for SERVER-4892.
Branch: master
https://github.com/mongodb/mongo/commit/26bb7661ebf0b58f5944ccfdfdf17127ab01819b

Comment by Andy Schwerin [ 13/Feb/12 ]

Tests written, but to be checked in.

Comment by auto [ 13/Feb/12 ]

Author:

{u'login': u'andy10gen', u'name': u'Andy Schwerin', u'email': u'schwerin@10gen.com'}

Message: SERVER-4892 - Perform correct authorization checks when doing killCursors.

Prior to this patch, we were incorrectly interpreting a portion of the dbKillCursors message as a
namespace string, for auth purposes. This was nonsense, as each cursor listed in the killCursors
could be from a distinct namespace. In this patch, the client's authority to kill each cursor is
checked as part of ClientCursor::erase. Failure to erase due to lack of authorization is reported
identically to failure because the cursor never existed or no longer exists.

Test to follow.
Branch: master
https://github.com/mongodb/mongo/commit/2d8237e652e2875fb829e9604f79fccb1643f241

Comment by Eliot Horowitz (Inactive) [ 10/Feb/12 ]

There are 2 other cases that are probably dupes, should make sure to clean those up with this is dealt with: SERVER-4600 and SERVER-626

One minor wrinkle is that drives right now do not guarantee that a killCursors is sent over the same connection that created it.

So we just have to make sure that still works as long as all connections are authenticated.

Comment by Christian Amor Kvalheim [ 09/Feb/12 ]

cool I'll be willing to test once the fix is in just tell me what branch to build.

Comment by Andy Schwerin [ 08/Feb/12 ]

Plan for addressing (1) above, fixing mongod, is to have ClientCursor::erase(CursorId) auth check before erasing, and skip the auth check for killCursors in assembleResponse.

Comment by Andy Schwerin [ 08/Feb/12 ]

There are multiple problems operating in concert, here.

1.) mongod incorrectly interprets the number-of-cursors field in the "kill cursors" message as the string namespace name. As a result, authentication of killCursors always fails. Mongod should instead be verifying that the internal object representing the client connection is authenticated against the cursor's associated namespace.

2.) Clients must either ensure that the connection across which they send "kill cursors" messages is authenticated for the namespaces associated with the condemned cursors, or they must getLastError to check for success when killing cursors.

I'll get to work on a solution for (1.), but the drivers will need an audit, if not an update.

Comment by Joseph Sofaer [ 07/Feb/12 ]

I am seeing this issue but would note that I can reproduce it with the app server and mongo server running on localhost on mac osx.

db version v2.0.2

Generated at Thu Feb 08 03:07:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.