[SERVER-76321] Buffer overrun while deserializing compound group key spilled to disk in SBE hash agg implementation Created: 19/Apr/23 Updated: 24/Jan/24 Resolved: 24/Apr/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 6.0.5, 6.3.0-rc3 |
| Fix Version/s: | 7.1.0-rc0, 6.0.6, 7.0.0-rc1, 6.3.2 |
| Type: | Bug | Priority: | Blocker - P1 |
| Reporter: | David Storch | Assignee: | David Storch |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Backport Requested: |
v7.0, v6.3, v6.0
|
||||||||||||
| Sprint: | QE 2023-05-01 | ||||||||||||
| Participants: | |||||||||||||
| Case: | (copied to CRM) | ||||||||||||
| Description |
|
Our current strategy for executing $group in most contexts is a hash aggregation. Namely, we maintain a hash table which maps from group key to accumulator state. In order to avoid using too much memory, when this hash table grows large enough it gets serialized to disk. This slot-based execution engine (SBE) implementation spills to an internal table managed by the storage engine using a type called TemporaryRecordStore. The keys of this spill table are serialized to an internal format called KeyString. We have discovered a buffer overrun bug related to the deserialization of KeyStrings to a MaterializedRow of SBE values. This can cause queries to fail with tassert() code 6136200. Given potential memory corruption, it could also lead to crashing with a segfault. A tassert() error message like the following one of the possible symptoms of this bug:
|
| Comments |
| Comment by Githook User [ 25/Apr/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: (cherry picked from commit bae7293f42adc498fe53d9a31e8f7fae07061e0c) |
| Comment by Githook User [ 25/Apr/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: (cherry picked from commit df1428dfe4fc4fa1dc7234aedce81344ebd9b609) |
| Comment by Githook User [ 24/Apr/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: (cherry picked from commit df1428dfe4fc4fa1dc7234aedce81344ebd9b609) |
| Comment by Githook User [ 24/Apr/23 ] |
|
Author: {'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}Message: |
| Comment by David Storch [ 19/Apr/23 ] |
|
I realized after some further experimentation that ObjectIds aren't actually necessary to trigger the tassert(). I'm adjusting the title, repro script, and description accordingly. |