[SERVER-59060] SBE replans far more aggressively than classic engine Created: 03/Aug/21  Updated: 29/Oct/23  Resolved: 18/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Ian Boros Assignee: Ian Boros
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: QE 2021-08-23
Participants:

 Description   

SBE will replan if the cached plan uses more works than the 'decisionReads'. See here. The classic engine only replans when the cached plan uses ~10x more works than expected. See here.

This could lead to scenarios where the query system repeatedly evicts cached plans when it isn't really necessary.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 17/Aug/21 ]

Author:

{'name': 'Ian Boros', 'email': 'ian.boros@mongodb.com', 'username': 'borosaurus'}

Message: SERVER-59060 Relax conditions under which SBE replans cached plans
Branch: master
https://github.com/mongodb/mongo/commit/7cf7ee6454842c3ade413ad305f0ac4ae8cefa25

Comment by Ian Boros [ 03/Aug/21 ]

Here's a script I used to test this. It could be simplified but I think it makes the point clear.

(function() {
    "use strict";
 
    const coll = db.sbe_replan;
    coll.createIndex({selectiveKey: 1});
    coll.createIndex({notSelectiveKey: 1});
 
    for (let i = 0; i < 10; ++i) {
        // Include one document which will not match the filter.
        assert.commandWorked(coll.insert({notSelectiveKey: 55, selectiveKey: i}));
 
        // If selectiveKey == 2 add one more document which doesn't match the filter.
        // This will make the plan using the index on selectiveKey a *little* bit less
        // 'productive', but not enough to warrant replanning. (< 1%).
        if (i == 2) {
            assert.commandWorked(coll.insert({notSelectiveKey: 55, selectiveKey: i}));
        }
 
        // Add a bunch of documents which do match the filter.
        for (let j = 0; j < 110; ++j) {
            assert.commandWorked(coll.insert({notSelectiveKey: 10, selectiveKey: i}));
        }
    }
 
    // Summary:
    // Possible values of selectiveKey: 0-9
    // Possible values of notSelectiveKey: [10, 55]
 
    // This should choose the plan using the index on 'selectiveKey'.
    printjson(coll.find({notSelectiveKey: {$lt: 50}, selectiveKey: 3}).explain());
    
    // Run some queries. The plan using the selectiveKey index should get cached.
    print("Running initial query\n");
    print(coll.find({notSelectiveKey: {$lt: 50}, selectiveKey: 3}).itcount());
    print(coll.find({notSelectiveKey: {$lt: 50}, selectiveKey: 3}).itcount());
    print(coll.find({notSelectiveKey: {$lt: 50}, selectiveKey: 3}).itcount());
    
    // Sleep to avoid overlapping debug prints etc.
    sleep(1000);
    print("Running new query\n");
    sleep(1000);
    // Run a query with the special value of 'selectiveKey' which has one extra document that
    // doesn't match the filter. The plan is indeed less 'efficient' but it's not
    // doing anywhere close to 10x more reads compared to the original plan.
    // This query gets replanned.
    print(coll.find({notSelectiveKey: {$lt: 50}, selectiveKey: 2}).itcount());
})();

Generated at Thu Feb 08 05:46:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.