[SERVER-84722] Create undocumented server parameter to skip document validation on insert code path for internal usage Created: 10/Jan/24  Updated: 31/Jan/24  Resolved: 26/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.6, 5.0.25, 4.4.29, 6.0.14, 8.0.0-rc0, 7.3.0-rc2

Type: Task Priority: Major - P3
Reporter: Ali Mir Assignee: Ali Mir
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-84734 Oplog entries can come close to 16 MB... Open
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.3, v7.0, v6.0, v5.0, v4.4
Sprint: Repl 2024-01-22, Repl 2024-02-05
Participants:
Case:

 Description   

In the server, the maximum BSON size for user documents is 16mb. Internally, we have a maximum size of 16mb + 16kb for internal documents that need to be slightly larger than the user maximum. An example is the oplog: when users insert a document that's close to 16MB in size, the corresponding oplog entry may exceed the user maximum to account for oplog entry metadata.

In the Cloud backup restore process, a PIT restore manually inserts oplog entries from a source cluster into a target cluster. If the oplog entries generated on the source cluster are sufficiently large, the oplog entries that we try to insert into the target cluster may be larger than 16mb. In that case, since the insert is happening via the regular insert codepath, it will fail validation here. This poses a problem for the restore process, as Cloud cannot get past these large documents. Users will be required to PIT restore to an earlier timestamp, or use a snapshot that already includes the large document, which may be undesirable.

In the server, we already have internal places where we skip document size validation, on account of oplog entries potentially exceeding the size limit. We should expose an undocumented server parameter to allow Cloud to skip document validation, specifically to address this edge case.



 Comments   
Comment by Githook User [ 31/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

(cherry picked from commit 7b27c54d39d8146fc31fd1959d92df78e75d4f2b)

GitOrigin-RevId: 9ebbf2ce73a95e3f1437e35065e59c038574e2f6
Branch: v4.4
https://github.com/mongodb/mongo/commit/cd53d98e3d09d5429ea38775a5166d1d21b5a42d

Comment by Githook User [ 31/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

(cherry picked from commit 7b27c54d39d8146fc31fd1959d92df78e75d4f2b)

GitOrigin-RevId: 1335fd397c658ec8c0f429f12dc27c85a33f516a
Branch: v5.0
https://github.com/mongodb/mongo/commit/02200f6f7895f2a4f647891f7bf0f5e67df8ae0d

Comment by Githook User [ 30/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

(cherry picked from commit 7b27c54d39d8146fc31fd1959d92df78e75d4f2b)

GitOrigin-RevId: 77d77b0781ab0697e7454c59f8da798813239c6b
Branch: v7.3
https://github.com/mongodb/mongo/commit/1db1781b4a707882e6308316ae0d25cdbf01451a

Comment by Githook User [ 30/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

(cherry picked from commit 7b27c54d39d8146fc31fd1959d92df78e75d4f2b)

GitOrigin-RevId: 76e53c9b8451d9530c5159ff55e2a06ab73e8f73
Branch: v7.0
https://github.com/mongodb/mongo/commit/a8d9c37c7351f3c68babf3476ccf42578643b01b

Comment by Githook User [ 30/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

(cherry picked from commit 7b27c54d39d8146fc31fd1959d92df78e75d4f2b)

GitOrigin-RevId: 35d7783145ab3160b95d95d4a540ebbe0bfb00d5
Branch: v6.0
https://github.com/mongodb/mongo/commit/3e4b71cb010e2ea21228d3d3328beb83d83db637

Comment by Githook User [ 26/Jan/24 ]

Author:

{'name': 'Ali Mir', 'email': 'ali.mir@mongodb.com', 'username': 'ali-mir'}

Message: SERVER-84722 Create internal server parameter to allow users to insert documents larger than the 16mb maximum user BSON size (#18255)

GitOrigin-RevId: 7b27c54d39d8146fc31fd1959d92df78e75d4f2b
Branch: master
https://github.com/mongodb/mongo/commit/c8f82a658c6b020839b1db97675da90df0a68380

Generated at Thu Feb 08 06:55:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.