-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Query Execution
-
Labels:None
-
ALL
-
-
Query Execution 2021-04-19
I've attached a repro script, which is just the generated test trimmed down to make it run much faster. The generated test came from this failing task. The offending query is the following:
[js_test:agg_fuzzer-8818-1618157020929-0] "command" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "aggregate" : "fuzzer_coll",
[js_test:agg_fuzzer-8818-1618157020929-0] "pipeline" : [
[js_test:agg_fuzzer-8818-1618157020929-0] {
[js_test:agg_fuzzer-8818-1618157020929-0] "$sort" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "_id" : 1
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] },
[js_test:agg_fuzzer-8818-1618157020929-0] {
[js_test:agg_fuzzer-8818-1618157020929-0] "$limit" : 20
[js_test:agg_fuzzer-8818-1618157020929-0] },
[js_test:agg_fuzzer-8818-1618157020929-0] {
[js_test:agg_fuzzer-8818-1618157020929-0] "$graphLookup" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "from" : "fuzzer_coll",
[js_test:agg_fuzzer-8818-1618157020929-0] "startWith" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "$setUnion" : [
[js_test:agg_fuzzer-8818-1618157020929-0] [
[js_test:agg_fuzzer-8818-1618157020929-0] /Dobra/
[js_test:agg_fuzzer-8818-1618157020929-0] ],
[js_test:agg_fuzzer-8818-1618157020929-0] "$array",
[js_test:agg_fuzzer-8818-1618157020929-0] "$obj.obj.obj.obj.array"
[js_test:agg_fuzzer-8818-1618157020929-0] ]
[js_test:agg_fuzzer-8818-1618157020929-0] },
[js_test:agg_fuzzer-8818-1618157020929-0] "connectFromField" : "obj.obj.obj.obj.obj.obj",
[js_test:agg_fuzzer-8818-1618157020929-0] "connectToField" : "obj.obj.obj.obj.date",
[js_test:agg_fuzzer-8818-1618157020929-0] "as" : "array",
[js_test:agg_fuzzer-8818-1618157020929-0] "depthField" : "obj.obj.num",
[js_test:agg_fuzzer-8818-1618157020929-0] "restrictSearchWithMatch" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "$or" : [
[js_test:agg_fuzzer-8818-1618157020929-0] {
[js_test:agg_fuzzer-8818-1618157020929-0] "obj.obj.obj.num" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "$not" : {
[js_test:agg_fuzzer-8818-1618157020929-0] "$bitsAnyClear" : 1001
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] ]
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] }
[js_test:agg_fuzzer-8818-1618157020929-0] ],
[js_test:agg_fuzzer-8818-1618157020929-0] "maxTimeMS" : 30000,
[js_test:agg_fuzzer-8818-1618157020929-0] "cursor" : {
[js_test:agg_fuzzer-8818-1618157020929-0]
[js_test:agg_fuzzer-8818-1618157020929-0] },
[js_test:agg_fuzzer-8818-1618157020929-0] "$db" : "fuzzer"
Interestingly, due to SERVER-56083, the query does not run using SBE in either the control (4.4) or experimental (5.0) versions. I suspect, however, that the queries executed recursively by $graphLookup end up using SBE, which is where things have gone wrong.
Looking at the output from the fuzzer, I can see that both servers returned 20 documents with the same _id values. However, none of the 20 documents returned by 4.4 were equal to any of the documents returned by 5.0. Drilling down, I looked specifically at the documents returned by each version with _id:0. I can see that the only difference between the two is the "array" field, which is what is produced by the $graphLookup stage.
For the document with _id:0, I dumped the contents of "array" from the lefthand-side server into a collection called "l" and the contents of "array" from the righthand-side server into a collection called "r", and this shows that the lengths of the arrays were different:
MongoDB Enterprise > db.l.count() 190 MongoDB Enterprise > db.r.count() 188
I believe this means that a few documents were missed by the $graphLookup when SBE was enabled. The following query shows exactly which were missed, specifically for the $graphLookup associated with _id:0:
MongoDB Enterprise > db.l.aggregate([{$lookup: {from: "r", as: "match", localField: "_id", foreignField: "_id"}}, {$match: {match: {$size: 0}}}]).pretty()
{
"_id" : 1078,
"str" : "orchid",
"num" : NumberLong(2178),
"date" : ISODate("2019-04-22T20:50:00.826Z"),
"array" : [
NumberLong(32525),
NumberLong(44047),
ISODate("2019-06-13T02:41:44.348Z"),
"Tools",
"Grove Product",
28213,
ISODate("2019-03-13T20:33:11.273Z"),
ISODate("2019-09-15T05:04:20.672Z"),
ISODate("2019-11-11T06:16:33.307Z"),
{
"_id" : 1079,
"str" : "Georgia Iraqi Dinar Lead",
"num" : 69643,
"date" : NaN,
"obj" : {
"_id" : 1080,
"str" : "generate THX virtual",
"num" : NumberLong(94448),
"array" : [
NumberDecimal("-643.826448748712"),
[ ],
NumberDecimal("272.28819709550726"),
null,
"e-tailers Mission"
],
"obj" : {
"_id" : 1081,
"str" : "envisioneer evolve Forks",
"num" : NumberDecimal("-397.33322040453436"),
"date" : ISODate("2019-07-05T14:28:07.010Z"),
"obj" : {
}
}
}
}
],
"obj" : {
"_id" : 1082,
"str" : "Intranet",
"num" : 74811,
"array" : [
{
"_id" : 1083,
"date" : ISODate("2019-05-15T16:46:30.286Z"),
"array" : [
{
"_id" : 1084,
"str" : null,
"num" : 20959,
"date" : null,
"array" : [ ],
"obj" : {
}
},
null,
NaN,
ISODate("2019-05-13T14:55:24.201Z")
]
}
],
"obj" : {
"_id" : 1085,
"array" : [
66718,
"back-end intangible",
[
"Multi-tiered turquoise Credit Card Account",
NumberDecimal("-4.27904291615323"),
{
"_id" : 1086,
"date" : ISODate("2019-10-15T15:08:13.433Z"),
"array" : [ ]
}
]
],
"obj" : {
"_id" : 1087,
"num" : NumberDecimal("-740.555235767481"),
"date" : ISODate("2019-11-23T19:12:50.139Z")
},
"num" : NumberLong(0)
}
},
"match" : [ ]
}
{
"_id" : 469,
"str" : "Tasty Rubber Pizza Bond Markets Units European Composite Unit (EURCO) robust",
"num" : NumberLong(81616),
"date" : ISODate("2019-08-05T01:32:15.829Z"),
"array" : [
ISODate("2019-01-28T14:08:07.624Z"),
"New Jersey Consultant",
23767,
72540,
ISODate("2019-12-28T19:30:44.661Z"),
NumberLong(32526),
"SQL",
[ ],
[ ],
75037
],
"obj" : {
"_id" : 470,
"str" : "Checking Account",
"num" : NumberLong(79901),
"date" : ISODate("2019-02-14T12:40:50.831Z"),
"array" : [
"CSS navigate Singapore",
null,
{
"_id" : 471,
"str" : "quantify",
"num" : NumberDecimal("-793.7706100048798"),
"date" : ISODate("2019-02-20T16:39:48.120Z"),
"array" : [
NaN,
NaN
],
"obj" : {
"_id" : 472,
"obj" : {
"_id" : 473,
"num" : 29622,
"date" : ISODate("2019-09-20T13:10:23.122Z"),
"array" : [
null,
ISODate("2019-10-15T02:26:25.924Z"),
[ ]
],
"obj" : {
}
}
}
}
],
"obj" : {
"_id" : 474,
"date" : NaN,
"array" : [ ],
"obj" : {
"_id" : 475,
"str" : "Mobility",
"num" : NumberDecimal("217.89759387808635")
},
"num" : NumberLong(0)
}
},
"match" : [ ]
}
- duplicates
-
-
- Closed
-
-
SERVER-53900 Ensure expression arguments checks semantics are the same between classic and SBE engines
-
- Closed
-
- is related to
-
SERVER-56083 [SBE] Expressions unsupported in SBE for agg operations prevent SBE engine from being selected even if not pushed down
-
- Closed
-