Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-68130

AutoSplitVector could generate response bigger than BSONObjMaxUserSize

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 6.0.1, 4.4.16, 4.2.22, 5.0.11, 6.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • ALL
    • v6.0, v5.0, v4.4, v4.2
    • Sharding EMEA 2022-08-08

      In HELP-35620 AutoSplitVector was called by the create collection coordinator and generated a response with too many split points. So when the response was serialized to a BSONObject the command failed with a BSONObjectTooLarge error.

      The AutoSplitVector has a logic in place to prevent this. In fact, while building the split points array, it tries to predict the final BSON object size and stops in case the maximum size is reached.

      As demonstrated in HELP-35620 this logic is somehow broken. From the customer's log we can see that the AutoSplitVector failed with BSONObjectTooLarge:

      2022-07-06T11:40:55.206-05:00 E  COMMAND  4879802 [conn124862] "Failed to handle request","attr":{"error":"BSONObjectTooLarge: BSON size limit hit while building Message. Size: 17973680 (0x11241B0); maxSize: 16793600(16MB)"}

      From the collection entry we can derive the ShardKey pattern for the collection:

      {"_id":"<collection.name>","lastmodEpoch":{"$oid":"62c70e4d59dd43b776579a8c"},"lastmod":{"$date":{"$numberLong":"1657212494323"}},"timestamp":{"$timestamp":{"t":1657212495,"i":218}},"uuid":{"$binary":{"base64":"pMa3VhfSS2ChFp59zYnnTg==","subType":"04"}},"key":{"ne_unique_id":{"$numberInt":"1"},"area-2":{"$numberInt":"1"},"cnum":{"$numberInt":"1"},"timestamp":{"$numberInt":"1"}},"unique":true,"noBalance":false}

      And from the chunks we can derive a possible split point structure:

      {
        "_id": {
          "$oid": "62c740258c72e513f017a59c"
        },
        "uuid": {
          "$binary": {
            "base64": "pMa3VhfSS2ChFp59zYnnTg==",
            "subType": "04"
          }
        },
        "min": {
          "ne_unique_id": "AFK_247123",
          "area-2": {
            "$numberInt": "247"
          },
          "cnum": {
            "$numberInt": "45"
          },
          "timestamp": {
            "$date": {
              "$numberLong": "1627797600000"
            }
          }
        },
        "max": {
          "ne_unique_id": "AFK_247123",
          "area-2": {
            "$numberInt": "247"
          },
          "cnum": {
            "$numberInt": "55"
          },
          "timestamp": {
            "$date": {
              "$numberLong": "1636117200000"
            }
          }
        },
        "shard": "myShard_0",
        "lastmod": {
          "$timestamp": {
            "t": 1540,
            "i": 40
          }
        },
        "history": [
          {
            "validAfter": {
              "$timestamp": {
                "t": 1657212495,
                "i": 218
              }
            },
            "shard": "myShard_0"
          }
        ]
      }

            Assignee:
            pierlauro.sciarelli@mongodb.com Pierlauro Sciarelli
            Reporter:
            tommaso.tocci@mongodb.com Tommaso Tocci
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: