Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-54379

[SBE] Diagnose 10% regression for the mixed_workloads and change_streams_throughput benchmark suites

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Querying
    • Labels:
    • Query Execution 2021-05-03

      The findOne() commands from the 0_1c_findOne benchmark (from the change_streams_throughput suite) and the mixed_findOne benchmark (from the mixed_workloads suite) are about 10% slower when SBE is enabled (compared with the classic execution engine).

      Instructions on how to run the findOne() command:

      1) Start mongod. If you want to enabled SBE, pass '--setParameter "featureFlagSBE=true"' on the command line.

      2) Launch the mongo shell and run the following commands to prepare your environment:

      > Random.setRandomSeed(0);
      > db.coll.drop();
      > db.coll.createIndex({id: 1});
      > db.coll.createIndex({a: 1});
      > let x47 = "x".repeat(47);
      > for (let i = 0; i < 1000; ++i) {
          let bulk = db.coll.initializeUnorderedBulkOp();
          for (let j = 0; j < 100; ++j) {
            bulk.insert(
              {id: Random.randInt(100000), a: Random.randInt(1000000), c: x47});
          }
          bulk.execute();
        }
      > function millis(command) { const t1 = new Date(); const result = command(); const t2 = new Date(); print("time: " + (t2 - t1) + "ms"); return (t2-t1); }

      3) Run the following commands multiple times to measure the performance of the find() commands:

      > millis(() => { for (let i = 0; i < 10000; ++i) { db.coll.findOne({a: i}); } })

      Here is how long 10,000 invocations of the findOne() command above took on my virtual workstation when SBE was enabled vs. when SBE was disabled (as of 4/23/2021):
        3310ms (SBE) vs. 2997ms (classic)

      Note: The example above will print out the total amount of time that 10,000 invocations of the "findOne(..)" shell command take from the client's perspective. If you are interested in how long each individual mongo command takes on the server, consider using the slow query timer.

            Assignee:
            mindaugas.malinauskas@mongodb.com Mindaugas Malinauskas
            Reporter:
            andrew.paroski@mongodb.com Drew Paroski
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: