[SERVER-8433] Aggregating deeply-nested documents can cause stack overflow Created: 01/Feb/13  Updated: 24/Jan/24  Resolved: 21/Mar/17

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Security, Stability
Affects Version/s: 2.3.2
Fix Version/s: 3.4.4, 3.5.5

Type: Bug Priority: Major - P3
Reporter: J Rassi Assignee: Kyle Suarez
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-13878 Issue with aggregation pipeline Closed
Related
related to SERVER-15559 Fatal Exception: Deeply nested $cond ... Closed
related to SERVER-6462 stack overflow when comparing deeply ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Backport Requested:
v3.4
Sprint: Query 2017-01-23, Query 2017-02-13, Query 2017-03-27
Participants:

 Description   

Recursive calls made to mongo::Document::toBson, in mongo/db/pipeline/document.cpp.

To reproduce (on my OS X 10.8 machine, triggers if levels >= 811):

function makeNestObj(depth){
    toret = {s : 1};
    for(i = 1; i < depth; i++){
        toret = {s : toret};
    }
    return toret;
}
db.foo.insert(makeNestObj(levels));
db.foo.aggregate({$project:{s:1}})

(snippet above borrowed from SERVER-6462)



 Comments   
Comment by Githook User [ 28/Mar/17 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'kyle.suarez@mongodb.com'}

Message: SERVER-8433 limit recursion when serializing Document to BSON

(cherry picked from commit b25e825ede4acd47bfdae25edef44c497ee2233e)

Conflicts:
src/mongo/db/pipeline/document_value_test.cpp
Branch: v3.4
https://github.com/mongodb/mongo/commit/1136f69ed91638712ab265fd3aff4c9bf3da5d10

Comment by Githook User [ 28/Mar/17 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'kyle.suarez@mongodb.com'}

Message: SERVER-8433 prohibit constructing FieldPaths that exceed the depth limit

(cherry picked from commit ba8c4a2cb599861bcd92446926b72bb17eb5df6b)

Conflicts:
src/mongo/db/pipeline/document_source_add_fields_test.cpp
src/mongo/db/pipeline/document_source_project_test.cpp
src/mongo/db/pipeline/field_path_test.cpp
Branch: v3.4
https://github.com/mongodb/mongo/commit/5630bd7769d5d03d017875c95ad2a6af9cbf90c0

Comment by Githook User [ 21/Mar/17 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'kyle.suarez@mongodb.com'}

Message: SERVER-8433 limit recursion when serializing Document to BSON
Branch: master
https://github.com/mongodb/mongo/commit/b25e825ede4acd47bfdae25edef44c497ee2233e

Comment by Githook User [ 21/Mar/17 ]

Author:

{u'username': u'ksuarz', u'name': u'Kyle Suarez', u'email': u'kyle.suarez@mongodb.com'}

Message: SERVER-8433 prohibit constructing FieldPaths that exceed the depth limit
Branch: master
https://github.com/mongodb/mongo/commit/ba8c4a2cb599861bcd92446926b72bb17eb5df6b

Comment by Tad Marshall [ 25/Jul/13 ]

If you hit the C++ stack overflow on the Mac but not in Linux at some specific depth, this is very likely to be SERVER-7735. The Mac gets a 512 KB stack for each thread instead of the 1024 KB stack that is used on Linux, so it will tend to hit stack overflow more quickly.

V8 has its own JavaScript stack which can also overflow. I suspect that in the first example ("db.foo.insert(makeNestObj(15000));") we never get to the point of passing the deep object to the Mongo code since the JS stack overflows before that can happen.

Every stack is going to have some limit, so to prevent overflows completely we'd need to use a non-recursive method to walk through a deeply nested structure.

Comment by Duraid Madina [ 25/Jul/13 ]

@Tad: no, we don't. levels=1000 works for me on linux (ulimit -s = 8MB)

interestingly, a deep js stack gets caught:

> db.foo.insert(makeNestObj(15000));
Thu Jul 25 15:47:51.725 JavaScript execution failed: RangeError: Maximum call stack size exceeded at src/mongo/shell/collection.js:L142

but we can certainly overflow our stack:

> db.foo.insert(makeNestObj(10000));
> db.foo.aggregate({$project:{s:1}})
Thu Jul 25 15:48:13.901 DBClientCursor::init call() failed
Thu Jul 25 15:48:13.902 JavaScript execution failed: Error: error doing query: failed at src/mongo/shell/query.js:L78
Thu Jul 25 15:48:13.902 trying reconnect to 127.0.0.1:27017

I ran into this with a similar aggregation (ending in thousands of ]}]}}]}]} ), the overflow occurred in the parser:

#824 0x0000000000867896 in mongo::Expression::parseObject (pBsonElement=pBsonElement@entry=0x7f10f5817ce0, pCtx=pCtx@entry=0x7f10fb96a580) at src/mongo/db/pipeline/expression.cpp:164
#825 0x000000000086805f in mongo::Expression::parseOperand (pBsonElement=0x7f10f5817ce0) at src/mongo/db/pipeline/expression.cpp:385
#826 0x0000000000869d62 in mongo::Expression::parseExpression (pOpName=<optimized out>, pBsonElement=<optimized out>) at src/mongo/db/pipeline/expression.cpp:357
#827 0x0000000000867896 in mongo::Expression::parseObject (pBsonElement=pBsonElement@entry=0x7f10f5817be0, pCtx=pCtx@entry=0x7f10fb96a940) at src/mongo/db/pipeline/expression.cpp:164

Comment by Tad Marshall [ 01/Feb/13 ]

Do we hit the stack overflow at the same point in Linux? If we hit it sooner in Mac OS X, this could be related to SERVER-7735.

Generated at Thu Feb 08 03:17:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.