[SERVER-61184] Compound 2dsphere indexes can generate a massive number of index keys per document Created: 02/Nov/21  Updated: 29/Oct/23  Resolved: 22/Dec/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0

Type: Bug Priority: Major - P3
Reporter: Henrik Edin Assignee: Dan Larkin-York
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-14995 [SERVER] Compound 2dsphere indexes ca... Closed
Backwards Compatibility: Minor Change
Operating System: ALL
Sprint: Execution Team 2021-11-29, Execution Team 2021-12-13, Execution Team 2021-12-27
Participants:
Case:

 Description   

When generating keys for 2dsphere indexes we calculate the cartesian products of all keys.

If documents contain locations with a large number of location coordinates compounded with an array of a large number of elements the resulting cartesian product can be massive in size.

When generating all these keys they are inserted into a single std::vector that needs to hold the cartesian product number of elements which can fail as the number is simply too large.



 Comments   
Comment by Dan Larkin-York [ 22/Dec/21 ]

This change does come with a minor change regarding backwards compatibility, in that previously the index build or document insertion would succeed as long as we had enough memory to build the keys and insert them. We will now fail the operation if a document would generate in excess of 100000 keys. This should impact few users, and can be worked around by increasing the value of the server parameter 'indexMaxNumGeneratedKeysPerDocument' if the server has enough memory to handle it.

Comment by Githook User [ 22/Dec/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-61184 Add limit to number of compound 2dsphere index keys we may generate

Co-authored-by: Henrik Edin<henrik.edin@mongodb.com>
Branch: master
https://github.com/mongodb/mongo/commit/af183c9e3f8f498c89440e28c949f4079ee37b39

Comment by Dianna Hohensee (Inactive) [ 02/Nov/21 ]

We think there needs to be some limits here, so users can't shoot themselves in the foot.

Alternatively: might be a bug that this is managing to happen / allowed.

Generated at Thu Feb 08 05:51:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.