Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.6.1
    • Labels:
      None
    • # Replies:
      4
    • Last comment by Customer:
      true

      Description

      I noticed while testing a different fix that the log-append.wtperf configuration ran excruciatingly slow (about 400 ops/second). I discovered that with that configuration every auto-txn is running fully synchronous even though the config and wtperf program take all the defaults (which should be for in-memory logging).

        Issue Links

          Activity

          Hide
          sue.loverso Sue LoVerso added a comment -

          The reason they're running slow is because the default in the wiredtiger_open call for transaction_sync is enabled=false,method=fsync. By setting enabled=false the log writes should not be synchronous until someone uses a transaction with sync=true. With enabled=false, the WT_LOG_FLUSH flag is cleared. But with method=fsync we set WT_LOG_FSYNC in the connection's txn_logsync field.

          The code in txn.i causes the txn to inherit the connection's setting and then modifies it based on the sync configuration in __wt_txn_config. If WT_LOG_FLUSH is not set and sync isn't true then we clear the txn_logsync field.

          However, that is not what is happening in the case of auto-transactions. The code inherits from the connection only, therefore, WT_LOG_FSYNC is set in the txn. The reason is that auto-transactions always send in a NULL cfg and the code never calls __wt_txn_config at all. Therefore the field is never cleared in this default case.

          I have a fix I'm testing.

          Show
          sue.loverso Sue LoVerso added a comment - The reason they're running slow is because the default in the wiredtiger_open call for transaction_sync is enabled=false,method=fsync . By setting enabled=false the log writes should not be synchronous until someone uses a transaction with sync=true . With enabled=false , the WT_LOG_FLUSH flag is cleared. But with method=fsync we set WT_LOG_FSYNC in the connection's txn_logsync field. The code in txn.i causes the txn to inherit the connection's setting and then modifies it based on the sync configuration in __wt_txn_config . If WT_LOG_FLUSH is not set and sync isn't true then we clear the txn_logsync field. However, that is not what is happening in the case of auto-transactions. The code inherits from the connection only, therefore, WT_LOG_FSYNC is set in the txn. The reason is that auto-transactions always send in a NULL cfg and the code never calls __wt_txn_config at all. Therefore the field is never cleared in this default case. I have a fix I'm testing.
          Hide
          sue.loverso Sue LoVerso added a comment -

          With this change, I'm getting 50000 ops/sec, versus 400 ops/sec before.

          Show
          sue.loverso Sue LoVerso added a comment - With this change, I'm getting 50000 ops/sec, versus 400 ops/sec before.
          Hide
          sue.loverso Sue LoVerso added a comment -

          FTR, I wondered why, with this change, the Jenkins plot didn't change at all for the log-append job. The reason is that although the individual job appears to use all the defaults in that config file, Jenkins overall sets an environment variable that turns off all syncing via transaction_sync=(method=none) for all jobs. Took me a while to understand why my identically configured AWS box was behaving different than the Jenkins job.

          Show
          sue.loverso Sue LoVerso added a comment - FTR, I wondered why, with this change, the Jenkins plot didn't change at all for the log-append job. The reason is that although the individual job appears to use all the defaults in that config file, Jenkins overall sets an environment variable that turns off all syncing via transaction_sync=(method=none) for all jobs. Took me a while to understand why my identically configured AWS box was behaving different than the Jenkins job.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

          Message: If we have no config, explicitly check for flushing and clear
          txn_logsync. WT-1949
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/d6f8320a2e5c6fcb665dd1b744dfe0474a8fb431

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'} Message: If we have no config, explicitly check for flushing and clear txn_logsync. WT-1949 Branch: develop https://github.com/wiredtiger/wiredtiger/commit/d6f8320a2e5c6fcb665dd1b744dfe0474a8fb431

            People

            • Assignee:
              sue.loverso Sue LoVerso
              Reporter:
              sue.loverso Sue LoVerso
              Participants:
              Last commenter:
              Michael Cahill
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                2 years, 2 weeks, 2 days ago
                Date of 1st Reply: