$planCacheStats errors due to large size of cache entry

XMLWordPrintableJSON

    • Query Optimization
    • Fully Compatible
    • ALL
    • Hide

      Here's a repro that demonstrates this issue in the presence of large $in lists:

      // Some comments here
function toMB(bytes) {
    return bytes / (1024 * 1024);
}

const coll = db.a;
coll.drop();

// Competing indexes so we actually cache.
coll.createIndex({a: 1});
coll.createIndex({b: 1});

coll.insert({a: 1});

const listSize = 1000000;
const inList = [];

for (let i = 0; i < listSize; i++) {
    inList.push(i + 1);
}

const query = {
    a: {$in: inList},
    b: 1,
};

print("query BSON size in mb: " + toMB(bsonsize(query)));

// Cache insertion.
for (let i = 0; i < 3; i++) {
    coll.find(query).toArray();
}

print("Finished running queries");

// We expect to fail at this aggregation.
const output = coll.aggregate([{$planCacheStats: {}}]).toArray();
print("output BSON size in mb: " + toMB(bsonsize(output)));

      The output on both 7.0 and master is:

      mongosh --port 27000 --file large_in_repro.js
DeprecationWarning: Collection.insert() is deprecated. Use insertOne, insertMany, or bulkWrite.
query BSON size in mb: 11.338157653808594
Finished running queries
MongoServerError: Executor error during aggregate command on namespace: test.a :: caused by :: Serializing Document failed :: caused by :: Size 35668515 exceeds maximum 16793600
      Show
      Here's a repro that demonstrates this issue in the presence of large $in lists: // Some comments here function toMB(bytes) { return bytes / (1024 * 1024); } const coll = db.a; coll.drop(); // Competing indexes so we actually cache. coll.createIndex({a: 1}); coll.createIndex({b: 1}); coll.insert({a: 1}); const listSize = 1000000; const inList = []; for (let i = 0; i < listSize; i++) { inList.push(i + 1); } const query = { a: {$ in : inList}, b: 1, }; print( "query BSON size in mb: " + toMB(bsonsize(query))); // Cache insertion. for (let i = 0; i < 3; i++) { coll.find(query).toArray(); } print( "Finished running queries" ); // We expect to fail at this aggregation. const output = coll.aggregate([{$planCacheStats: {}}]).toArray(); print( "output BSON size in mb: " + toMB(bsonsize(output))); The output on both 7.0 and master is: mongosh --port 27000 --file large_in_repro.js DeprecationWarning: Collection.insert() is deprecated. Use insertOne, insertMany, or bulkWrite. query BSON size in mb: 11.338157653808594 Finished running queries MongoServerError: Executor error during aggregate command on namespace: test.a :: caused by :: Serializing Document failed :: caused by :: Size 35668515 exceeds maximum 16793600
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      If a given plan cache entry is too large, for example due to having a large $in list and therefore, a lot of index bounds, we can run into the BSON size limit and error during execution of the $planCacheStats stage. This severely hampers debuggability as seen in HELP-89371 since we cannot inspect the plan cache.

      The error message is:

      MongoServerError: Executor error during aggregate command on namespace: test.a :: caused by :: Serializing Document failed :: caused by :: Size 35668515 exceeds maximum 16793600

            Assignee:
            Alexander Ignatyev
            Reporter:
            Naafiyan Ahmed
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: