[SERVER-75261] "listCollections" command fails with BSONObjectTooLarge error Created: 24/Mar/23  Updated: 29/Oct/23  Resolved: 30/Mar/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.15, 6.0.5
Fix Version/s: 7.0.0-rc0, 6.0.6, 5.0.17, 4.4.21, 6.3.1

Type: Bug Priority: Major - P3
Reporter: Mindaugas Malinauskas Assignee: Mindaugas Malinauskas
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.3, v6.0, v5.0, v4.4, v4.2
Steps To Reproduce:

(function() {
"use strict";
 
const validatorObj = {
$jsonSchema: {
bsonType: "object",
properties: {
s: {bsonType: "string", description: "x".repeat(4801)},
 
}
}
};
 
const nCollections = 3300;
assert.commandWorked(db.dropDatabase());
jsTestLog(`Creating ${nCollections} collections....`);
for (let i = 0; i < nCollections; i++) {
assert.commandWorked(db.createCollection("c_" + i.toPrecision(6), {"validator": validatorObj}));
}
jsTestLog(`Done creating ${nCollections} collections`);
const res = db.runCommand({"listCollections": 1});
assert.commandWorked(res);
}());

Sprint: QE 2023-04-03
Participants:

 Description   

"listCollections" command fails with BSONObjectTooLarge error when:

  1. the batch size is not specified or is large enough;
  2. the number of entries to return is larger than approximately 3300;
  3. total BSON size of entries to return approaches 16MB.

Likely other commands such as "find" and "aggregate" are affected.

Extremely likely the root cause is not accounting for array element field element size when building the result.



 Comments   
Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-75261 Added accounting for array element overhead for "listCollections", "listIndexes", "_shardsvrCheckMetadataConsistencyParticipant" commands

(cherry picked from commit 3cde7fd5d90f1e6bd16d38cc668963a14671d690)
(cherry picked from commit 9cd5eeeba2184ce807985438ee984d7d8485b224)
(cherry picked from commit ba334a1cb5f1a209e4f811a24ef1a6a65a1c9d0c)
(cherry picked from commit be81ddbba83a2a5220376a35aa7f2ad070b78300)
Branch: v4.4
https://github.com/mongodb/mongo/commit/9d7cec04dc15c878b8a1e0368c7457d0fd144d03

Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-75261 Added accounting for array element overhead for "listCollections", "listIndexes", "_shardsvrCheckMetadataConsistencyParticipant" commands

(cherry picked from commit 3cde7fd5d90f1e6bd16d38cc668963a14671d690)
(cherry picked from commit 9cd5eeeba2184ce807985438ee984d7d8485b224)
(cherry picked from commit ba334a1cb5f1a209e4f811a24ef1a6a65a1c9d0c)
Branch: v5.0
https://github.com/mongodb/mongo/commit/be81ddbba83a2a5220376a35aa7f2ad070b78300

Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-75261 Added accounting for array element overhead for "listCollections", "listIndexes", "_shardsvrCheckMetadataConsistencyParticipant" commands

(cherry picked from commit 3cde7fd5d90f1e6bd16d38cc668963a14671d690)
(cherry picked from commit 9cd5eeeba2184ce807985438ee984d7d8485b224)
Branch: v6.0
https://github.com/mongodb/mongo/commit/ba334a1cb5f1a209e4f811a24ef1a6a65a1c9d0c

Comment by Githook User [ 12/Apr/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-75261 Added accounting for array element overhead for "listCollections", "listIndexes", "_shardsvrCheckMetadataConsistencyParticipant" commands

(cherry picked from commit 3cde7fd5d90f1e6bd16d38cc668963a14671d690)
Branch: v6.3
https://github.com/mongodb/mongo/commit/9cd5eeeba2184ce807985438ee984d7d8485b224

Comment by Githook User [ 29/Mar/23 ]

Author:

{'name': 'Mindaugas Malinauskas', 'email': 'mindaugas.malinauskas@mongodb.com', 'username': ''}

Message: SERVER-75261 Added accounting for array element overhead for "listCollections", "listIndexes", "_shardsvrCheckMetadataConsistencyParticipant" commands
Branch: master
https://github.com/mongodb/mongo/commit/3cde7fd5d90f1e6bd16d38cc668963a14671d690

Generated at Thu Feb 08 06:29:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.