A recent change to the transaction ID allocation code introduced a performance bottleneck and none of our automated performance tests noticed it. The change that introduced the performance regression was WT-3415.
The bottleneck was created when releasing a transaction ID, and the place that really notices that is an in cache update heavy workload.
I ran the following workload:
# wtperf options file: btree with inserts/updates, in memory conn_config="cache_size=10G" table_config="type=file" icount=5000000 report_interval=5 run_time=120 populate_threads=1 threads=((count=16,updates=1))
Against before and after the changes for WT-3415, and it showed a 40% performance regression - from 465960 ops/sec to 296158 ops/sec on my local server.
We should add that test case to Jenkins, so we detect similar performance regressions in the future.