[SERVER-79136] Incorrect query result of $match + $group on metaField over time-series Created: 19/Jul/23  Updated: 29/Oct/23  Resolved: 24/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 6.0.8, 7.0.0-rc8
Fix Version/s: 7.1.0-rc0, 5.0.20, 6.0.9, 7.0.0-rc9

Type: Bug Priority: Critical - P2
Reporter: Irina Yatsenko (Inactive) Assignee: Irina Yatsenko (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0, v5.0
Sprint: QI 2023-07-24, QI 2023-08-07
Participants:

 Description   

> ts.find()
{ "time" : ISODate("2023-07-19T20:37:09.316Z"), "meta" : 1, "val" : 5, "_id" : 0 }
{ "time" : ISODate("2023-07-19T20:37:31.758Z"), "meta" : 1, "val" : 21, "_id" : 1 }
 
// filter by the date of the document with _id: 1
> ts.aggregate([{$match: {time: ISODate("2023-07-19T20:37:31.758Z")}}])
{ "time" : ISODate("2023-07-19T20:37:31.758Z"), "meta" : 1, "val1" : 3, "val" : 21, "_id" : 1 }
 
// add group on the metaField after the filter
> ts.aggregate([{$match: {time: ISODate("2023-07-19T20:37:31.758Z")}},{$group: {_id: "$meta", m: {$min: "$val"}}}])
{ "_id" : 1, "m" : 5 }

The result of the last query is incorrect – it should have been {_id: 1, m: 21}.
The problem doesn't repro if group key isn't the metaField and does repro if filter on any non-metaField.

The problem is with the $group rewrite that replaces "$group": {"_id": "$meta", "m": {"$min": "$val2" }} with "$group": {"_id": "$meta", "m": {"$min": "$control.min.val2"}} without taking in account that the minimum should be computed over a subset of the bucket.



 Comments   
Comment by Githook User [ 31/Jul/23 ]

Author:

{'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}

Message: SERVER-79136 Block $group min/max rewrite in TS if there is a non-meta filter

(cherry picked from commit 1a6d2cc2f17e91f81714b009ee7840c1099f5440)
(cherry picked from commit 5cedf9a6870900e558b838414a3d0a436a4b0944)
(cherry picked from commit 3ef3bc5f66488b02b116513d24ff001991c2b38e)
Branch: v5.0
https://github.com/mongodb/mongo/commit/f1f5acbdcc8d44961a3a31550d4385b3ec8c0b9c

Comment by Githook User [ 27/Jul/23 ]

Author:

{'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}

Message: SERVER-79136 Block $group min/max rewrite in TS if there is a non-meta filter

(cherry picked from commit 1a6d2cc2f17e91f81714b009ee7840c1099f5440)
(cherry picked from commit 5cedf9a6870900e558b838414a3d0a436a4b0944)
Branch: v6.0
https://github.com/mongodb/mongo/commit/4defc357136e321f8962efe2f6c34334b4581290

Comment by Irina Yatsenko (Inactive) [ 24/Jul/23 ]

There are two bugs in the same area here. One was introduced by https://jira.mongodb.org/browse/SERVER-67780 which fixed yet another incorrect query re-write and dates back to 5.0. The other one might have been in since the original implementation of the group rewrites that were added to 5.0 per SERVER-55575.

Comment by Githook User [ 24/Jul/23 ]

Author:

{'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}

Message: SERVER-79136 Block $group min/max rewrite in TS if there is a non-meta filter

(cherry picked from commit 1a6d2cc2f17e91f81714b009ee7840c1099f5440)
Branch: v7.0
https://github.com/mongodb/mongo/commit/0cb7904e55f7be264a8de281f57fd0b4a03d5c71

Comment by Githook User [ 21/Jul/23 ]

Author:

{'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}

Message: SERVER-79136 Block $group min/max rewrite in TS if there is a non-meta filter
Branch: master
https://github.com/mongodb/mongo/commit/1a6d2cc2f17e91f81714b009ee7840c1099f5440

Generated at Thu Feb 08 06:40:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.