Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-68130

AutoSplitVector could generate response bigger than BSONObjMaxUserSize

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 6.0.1, 4.4.16, 4.2.22, 5.0.11, 6.1.0-rc0
    • None
    • None
    • Fully Compatible
    • ALL
    • v6.0, v5.0, v4.4, v4.2
    • Sharding EMEA 2022-08-08

    Description

      In HELP-35620 AutoSplitVector was called by the create collection coordinator and generated a response with too many split points. So when the response was serialized to a BSONObject the command failed with a BSONObjectTooLarge error.

      The AutoSplitVector has a logic in place to prevent this. In fact, while building the split points array, it tries to predict the final BSON object size and stops in case the maximum size is reached.

      As demonstrated in HELP-35620 this logic is somehow broken. From the customer's log we can see that the AutoSplitVector failed with BSONObjectTooLarge:

      2022-07-06T11:40:55.206-05:00 E  COMMAND  4879802 [conn124862] "Failed to handle request","attr":{"error":"BSONObjectTooLarge: BSON size limit hit while building Message. Size: 17973680 (0x11241B0); maxSize: 16793600(16MB)"}

      From the collection entry we can derive the ShardKey pattern for the collection:

      {"_id":"<collection.name>","lastmodEpoch":{"$oid":"62c70e4d59dd43b776579a8c"},"lastmod":{"$date":{"$numberLong":"1657212494323"}},"timestamp":{"$timestamp":{"t":1657212495,"i":218}},"uuid":{"$binary":{"base64":"pMa3VhfSS2ChFp59zYnnTg==","subType":"04"}},"key":{"ne_unique_id":{"$numberInt":"1"},"area-2":{"$numberInt":"1"},"cnum":{"$numberInt":"1"},"timestamp":{"$numberInt":"1"}},"unique":true,"noBalance":false}

      And from the chunks we can derive a possible split point structure:

      {
        "_id": {
          "$oid": "62c740258c72e513f017a59c"
        },
        "uuid": {
          "$binary": {
            "base64": "pMa3VhfSS2ChFp59zYnnTg==",
            "subType": "04"
          }
        },
        "min": {
          "ne_unique_id": "AFK_247123",
          "area-2": {
            "$numberInt": "247"
          },
          "cnum": {
            "$numberInt": "45"
          },
          "timestamp": {
            "$date": {
              "$numberLong": "1627797600000"
            }
          }
        },
        "max": {
          "ne_unique_id": "AFK_247123",
          "area-2": {
            "$numberInt": "247"
          },
          "cnum": {
            "$numberInt": "55"
          },
          "timestamp": {
            "$date": {
              "$numberLong": "1636117200000"
            }
          }
        },
        "shard": "myShard_0",
        "lastmod": {
          "$timestamp": {
            "t": 1540,
            "i": 40
          }
        },
        "history": [
          {
            "validAfter": {
              "$timestamp": {
                "t": 1657212495,
                "i": 218
              }
            },
            "shard": "myShard_0"
          }
        ]
      }

      Attachments

        Activity

          People

            pierlauro.sciarelli@mongodb.com Pierlauro Sciarelli
            tommaso.tocci@mongodb.com Tommaso Tocci
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: