Summary:
Fix problems where calling the WiredTiger API in unexpected ways could lead to incorrect results. Specifically, calling unsupported functions after a prepare could lead to transactions being rolled back, even if commit was called, or leaving a transaction unresolved until the session was closed if commit was called when rollback should have been called.
===================-=
Over the weekend while trying to reproduce WT-5119, I hit an assertion failure on kodkod-aws. This might be related to WT-5318, but it isn't clear. This occurred on develop (as of mid-November), changeset 07b88355.
I see this failure:
[1576398236:585956][18160:0x7f5c4c9bc700], t, WT_SESSION.commit_transaction: __session_commit_transaction, 1649: failed transaction requires rollback: oldest pinned transaction ID rolled back for eviction: Invalid argument [1576398236:585988][18160:0x7f5c4c9bc700], t, WT_SESSION.commit_transaction: __session_commit_transaction, 1659: WT_SESSION_TXN_STATE(session)->id == WT_TXN_NONE
The transaction's id is not WT_TXN_NONE but a real value:
(gdb) p session->txn $1 = {id = 38733, isolation = WT_ISO_SNAPSHOT, forced_iso = 0, snap_min = 38718, snap_max = 38718, snapshot = 0x7f5c344db140, snapshot_count = 0, txn_logsync = 4, commit_timestamp = 55334, durable_timestamp = 55334, first_commit_timestamp = 55334, prepare_timestamp = 0, read_timestamp = 55333, durable_timestampq = {tqe_next = 0x0, tqe_prev = 0x1ac9f48}, read_timestampq = { tqe_next = 0x0, tqe_prev = 0x1ac9f88}, clear_durable_q = false, clear_read_q = false, mod = 0x7f5c342317a0, mod_alloc = 1120, mod_count = 1, logrec = 0x7f5c3438cd90, notify = 0x0, ckpt_lsn = {l = {offset = 0, file = 0}, file_offset = 0}, ckpt_nsnapshot = 0, ckpt_snapshot = 0x0, full_ckpt = false, operation_timeout_us = 0, rollback_reason = 0x61fd58 "oldest pinned transaction ID rolled back for eviction", flags = 2107550}