[JAVA-293] Cursor is not closed after limit is fully fetched from server Created: 08/Mar/11 Updated: 17/Mar/11 Resolved: 09/Mar/11 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | API |
| Affects Version/s: | 2.4 |
| Fix Version/s: | 2.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | John Ayres | Assignee: | Antoine Girbal |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
We're trying to load all docs from a collection in batches of 100 (to avoid locking the DB for a long period of time). Our objects are large and this eventually causes the MongoDB server to run out of memory. I've run a test locally to recreate the problem and it seems that cursors on the server are not being closed. When running the code below the number of open cursors on the server gradually increases. int batchSize = 100; The output of db.serverStatus() after about 1 min: "cursros" : { "totalOpen" : 51, "clientCursors_size" : 51, "timedOut" : 0 }, It appears that using a negative limit fixes this, but there appears to be a 4MB upper limit of the number of docs returned. Why does the server keep the cursor open when all the results have been read? |
| Comments |
| Comment by auto [ 09/Mar/11 ] |
|
Author: {u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}Message: |
| Comment by Antoine Girbal [ 09/Mar/11 ] |
|
This turned out to be more of a refactoring than hoped, to clean up code while fixing issue.
this mimic the underlying variable of the protocol, so will be easier to support. Note that for explain(), the value of limit has to be made negative (guess that's how server needs it). |
| Comment by auto [ 09/Mar/11 ] |
|
Author: {u'login': u'agirbal', u'name': u'agirbal', u'email': u'antoine@10gen.com'}Message:
|
| Comment by Antoine Girbal [ 08/Mar/11 ] |
|
did some quick tests and looks like indeed we are not closing cursor when limit is involved. |