[SERVER-10554] memory leak in MutableDocument::setNestedField Created: 16/Aug/13  Updated: 11/Jul/16  Resolved: 19/Aug/13

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: 2.4.5
Fix Version/s: 2.4.7, 2.5.2

Type: Bug Priority: Major - P3
Reporter: Alexander Komyagin Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Win2008R2
3-node replica set


Issue Links:
Depends
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   
Issue Status as of October 23rd, 2013

ISSUE SUMMARY
Memory leak in aggregation in MutableDocument::setNestedField

USER IMPACT
This leak can manifest in two different scenarios:

  1. Accessing nested fields in the original document fetched from the collection. This leads to a leak of a few hundred bytes per aggregation command. The amount of memory leaked in this case does not depend on the amount of data processed in the aggregation. If the nested fields are only used by a $match expression but not any other pipeline stages, this leak does not apply.
  2. Using a $geoNear pipeline stage and using distanceField or includeLocs to inject information into a nested field. This results in a leak of a few hundred bytes per document returned from geoNear.

It is present in versions of MongoDB prior to and including v2.4.6.

SOLUTION
Fix ref counting of NULL pointers in MutableDocument.

WORKAROUNDS
None.

PATCHES
Production release v2.4.7 contains the fix for this issue, and production release v2.6.0 will contain the fix as well.

Original Description

Aggregation queries are causing memory leak on primary. Detected by comparing umdh memory dumps before and after running aggregation.

There are two cases in the current codebase that can cause this leak:

1) Accessing nested fields in the original document fetched from the collection. This leads to a leak of a few hundred bytes per aggregation command. The amount of memory leaked in this case does not depend on the amount of data processed in the aggregation. If the nested fields are only used by a $match expression but not any other pipeline stages, this leak does not apply.

2) Using a $geoNear pipeline stage and using distanceField or includeLocs to inject information into a nested field. This results in a leak of a few hundred bytes per documented returned from geoNear.



 Comments   
Comment by auto [ 23/Aug/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Add check to prevent future memory leaks like SERVER-10554
Branch: v2.4
https://github.com/mongodb/mongo/commit/26f638873eadfc3dd8e17bb37eb68a1c613146c3

Comment by auto [ 23/Aug/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-10554 Patch memory leak in aggregation

The issue was that empty documents are represented as a NULL pointer and
Value didn't ref count NULL pointers. This is fine for all public
interactions with Value, however MutableDocument can change the pointer
in-place when updating a nested field. This caused a leak when it
transitioned a Value from holding a non-refcounted NULL pointer to
holding a real pointer since MutableDocument/MutableValue didn't set the
refCount bit.
Branch: v2.4
https://github.com/mongodb/mongo/commit/cc9361ee7e40cb087d779eef7e10b2225c7ff8f7

Comment by auto [ 19/Aug/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: Add check to prevent future memory leaks like SERVER-10554
Branch: master
https://github.com/mongodb/mongo/commit/33342134b8a72434b4281296740ff14f60c20c77

Comment by auto [ 19/Aug/13 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'mathias@10gen.com'}

Message: SERVER-10554 Patch memory leak in aggregation

The issue was that empty documents are represented as a NULL pointer and
Value didn't ref count NULL pointers. This is fine for all public
interactions with Value, however MutableDocument can change the pointer
in-place when updating a nested field. This caused a leak when it
transitioned a Value from holding a non-refcounted NULL pointer to
holding a real pointer since MutableDocument/MutableValue didn't set the
refCount bit.
Branch: master
https://github.com/mongodb/mongo/commit/6fea4eb1774119e6431bf251d008d062c67c73f2

Generated at Thu Feb 08 03:23:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.