Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12803

Optimise where we can skip copying keys for prepared transaction resolution

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Storage Engines
    • StorEng - Defined Pipeline

      A regular transaction maintains an in-memory copy of each key that gets modified. These keys are kept in memory for the duration of the transaction. This is done as a part of __wt_txn_op_set_key(). If a transaction is not prepared, we discard these keys during commit/rollback. For the prepared transactions, if the corresponding changed pages get evicted, these in-memory keys are meant to be able to help search() the records on disk and bring them back into memory for the transaction resolution.

      Most transactions never go through the prepared stage. This extra key copy is followed by freeing the keys without ever using them. If we know in advance that a transaction will not go through a prepared stage, we can skip the key allocation and possibly improve performance.

      Anecdotally speaking, the server only calls prepare on those multi-doc transactions that have one or more shards involved. Not only are these cases rare, but I am told that the server knows these cases in advance before starting the WiredTiger transaction.

      With this ticket, we can explore performance improvements. I see the following steps:

      1. Check if a performance improvement is worthy of doing this work. luke.pearson@mongodb.com has a POC and perf-test patches to get started on.
      2. Validate if the server knows what transactions will be prepared before starting the storage transaction. What changes are needed in the server, and if given a configuration option to the transaction begin API, where and how would it be used?
        3. Is there any additional complexity in Server or Storage that needs to be weighed against potential performance improvements?

      cc. luke.pearson@mongodb.com, blake.oler@mongodb.com

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: