Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-84722

Create undocumented server parameter to skip document validation on insert code path for internal usage

    • Fully Compatible
    • v7.3, v7.0, v6.0, v5.0, v4.4
    • Repl 2024-01-22, Repl 2024-02-05

      In the server, the maximum BSON size for user documents is 16mb. Internally, we have a maximum size of 16mb + 16kb for internal documents that need to be slightly larger than the user maximum. An example is the oplog: when users insert a document that's close to 16MB in size, the corresponding oplog entry may exceed the user maximum to account for oplog entry metadata.

      In the Cloud backup restore process, a PIT restore manually inserts oplog entries from a source cluster into a target cluster. If the oplog entries generated on the source cluster are sufficiently large, the oplog entries that we try to insert into the target cluster may be larger than 16mb. In that case, since the insert is happening via the regular insert codepath, it will fail validation here. This poses a problem for the restore process, as Cloud cannot get past these large documents. Users will be required to PIT restore to an earlier timestamp, or use a snapshot that already includes the large document, which may be undesirable.

      In the server, we already have internal places where we skip document size validation, on account of oplog entries potentially exceeding the size limit. We should expose an undocumented server parameter to allow Cloud to skip document validation, specifically to address this edge case.

            ali.mir@mongodb.com Ali Mir
            ali.mir@mongodb.com Ali Mir
            0 Vote for this issue
            9 Start watching this issue