Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-92276

Improve mongos unordered write batching when the batch operation has a mix of targeted and broadcasted writes

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Catalog and Routing
    • 3

      Consider an ordered=false batch write consisting of a mix of operations that can be targeted to a single shard and operations that need to be broadcasted to multiple shards. E.g for a sharded collection with shard key x: 'hashed' and chunks on two different shards:

      {
        update: 'foo', 
        updates: [
          {q: {x: 1}, u: {$inc: {c1: 1}}, multi: false}, 
          {q: {}, u: {$set: {a: 1}}, multi: true}, 
          {q: {x: 1}, u: {$inc: {c2: 1}}, multi: false}
        ], 
        ordered: false
      }
      

      Currently, mongod will execute this as three different batches:

      1. {q: {x: 1}, u: {$inc: {c1: 1}}, multi: false} to shardA, with shardA's ShardVersion attached
      2. {q: {}, u: {$set: {a: 1}}, multi: true}       to [shardA, shardB], with ShardVersion::IGNORED attached
      3. {q: {x: 1}, u: {$inc: {c2: 1}}, multi: false} to shardA, with shardA's ShardVersion attached
      

      This happens because mongos scans the operation in order and breaks the batch as soon as it finds one operation that cannot be included in the current batch, in this case because the second operation is multi=true and it uses ShardVersion::IGNORED whereas the batch uses targeted shard versions.

      This is correct, but it can be improved. Operations 1 and 3 can be batched together because both are targeted to shardA. Later operation 2 can be executed on different batch. I.e.

      1. [{q: {x: 1}, u: {$inc: {c1: 1}}, multi: false}, {q: {x: 1}, u: {$inc: {c2: 1}}, multi: false}] to shardA, with shardA's ShardVersion attached
      2. {q: {}, u: {$set: {a: 1}}, multi: true} to [shardA, shardB],                                   to [shardA, shardB] with ShardVersion::IGNORED attached
      

      This will make batched writes with mixed targeted and broadcasted operations more performant.

            Assignee:
            Unassigned Unassigned
            Reporter:
            jordi.serra-torrens@mongodb.com Jordi Serra Torrens
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: