[SERVER-61184] Compound 2dsphere indexes can generate a massive number of index keys per document Created: 02/Nov/21 Updated: 29/Oct/23 Resolved: 22/Dec/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 5.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Henrik Edin | Assignee: | Dan Larkin-York |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Sprint: | Execution Team 2021-11-29, Execution Team 2021-12-13, Execution Team 2021-12-27 | ||||||||||||
| Participants: | |||||||||||||
| Case: | (copied to CRM) | ||||||||||||
| Description |
|
When generating keys for 2dsphere indexes we calculate the cartesian products of all keys. If documents contain locations with a large number of location coordinates compounded with an array of a large number of elements the resulting cartesian product can be massive in size. When generating all these keys they are inserted into a single std::vector that needs to hold the cartesian product number of elements which can fail as the number is simply too large. |
| Comments |
| Comment by Dan Larkin-York [ 22/Dec/21 ] |
|
This change does come with a minor change regarding backwards compatibility, in that previously the index build or document insertion would succeed as long as we had enough memory to build the keys and insert them. We will now fail the operation if a document would generate in excess of 100000 keys. This should impact few users, and can be worked around by increasing the value of the server parameter 'indexMaxNumGeneratedKeysPerDocument' if the server has enough memory to handle it. |
| Comment by Githook User [ 22/Dec/21 ] |
|
Author: {'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}Message: Co-authored-by: Henrik Edin<henrik.edin@mongodb.com> |
| Comment by Dianna Hohensee (Inactive) [ 02/Nov/21 ] |
|
We think there needs to be some limits here, so users can't shoot themselves in the foot. Alternatively: might be a bug that this is managing to happen / allowed. |