Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-44095

S2 key generation vector should use a smaller type

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.4.0-rc0, 4.7.0
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Fully Compatible
    • ALL
    • v4.4
    • Execution Team 2020-03-09, Execution Team 2020-03-23
    • 49

      S2, "2dsphere", indexes generate keys usingĀ a vector of KeyString::HeapBuilder. Unfortunately, this type is very large (608 bytes), as compared to the previously used BSONObj (16 bytes). This type is large because of the StackBufBuilder used for the TypeBits member, which when not allocated on the stack, adds an additional 512 bytes to the size of this type. Most of this space is unused.

      This causes problems for 2dsphere indexes, which when building a compound key where the first field is an array, need to allocate as many KeyString::HeapBuilders as there are elements in the array. If the array is very large, say 1500000 elements, then the minimum memory requirement to generate the keys is close to 900MB.

      We should find a way to use a smaller type in this vector, KeyString::Value (24 bytes) for example, and resort to copying it as necessary.

      An alternative would be to make TypeBits support the BufBuilder instead of the StackBufBuilder, but that would only reduce the HeapBuilder type to around 96 bytes.

            Assignee:
            louis.williams@mongodb.com Louis Williams
            Reporter:
            louis.williams@mongodb.com Louis Williams
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: