Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.7.0
    • Labels:
      None
    • # Replies:
      5
    • Last comment by Customer:
      true

      Description

      The Jenkins run of log-append.wtperf hung with all relevant threads yielding in log_slot_join but no other thread in mid-operation.

        Issue Links

          Activity

          Hide
          sue.loverso Sue LoVerso added a comment -

          Unfortunately the Jenkins executable does not have symbols in it so other than stacks I cannot see any content. I'm running it on my AWS HDD box in a loop to try to repro.

          This must be highly optimized because _wt_log_write_internal is the caller of _wt_log_slot_join and it is not present on the stack.

          Thread 8 (Thread 0x7f268d7f9700 (LWP 8452)):
          #0  0x00007f2693e1bb97 in sched_yield () from /lib64/libc.so.6
          #1  0x00000000004249c8 in __wt_log_slot_join ()
          #2  0x0000000000422219 in __wt_log_write ()
          #3  0x000000000045cf18 in __wt_txn_commit ()
          #4  0x0000000000409155 in populate_thread ()
          

          Show
          sue.loverso Sue LoVerso added a comment - Unfortunately the Jenkins executable does not have symbols in it so other than stacks I cannot see any content. I'm running it on my AWS HDD box in a loop to try to repro. This must be highly optimized because _ wt_log_write_internal is the caller of _wt_log_slot_join and it is not present on the stack. Thread 8 (Thread 0x7f268d7f9700 (LWP 8452)): #0 0x00007f2693e1bb97 in sched_yield () from /lib64/libc.so.6 #1 0x00000000004249c8 in __wt_log_slot_join () #2 0x0000000000422219 in __wt_log_write () #3 0x000000000045cf18 in __wt_txn_commit () #4 0x0000000000409155 in populate_thread ()
          Hide
          sue.loverso Sue LoVerso added a comment -

          Good news is I can reproduce this every 5 or 6 iterations. We close and write a slot, but a new slot is not installed.

          Show
          sue.loverso Sue LoVerso added a comment - Good news is I can reproduce this every 5 or 6 iterations. We close and write a slot, but a new slot is not installed.
          Hide
          sue.loverso Sue LoVerso added a comment -

          I understand this now. Similar to log switching if a forced write hits the EBUSY from newfile having to wait for the previous log file to close, it needs to retry. I'm going to look at combining the internals from switch and force write as they're very similar so that we only have to solve problems like this in one place.

          Show
          sue.loverso Sue LoVerso added a comment - I understand this now. Similar to log switching if a forced write hits the EBUSY from newfile having to wait for the previous log file to close, it needs to retry. I'm going to look at combining the internals from switch and force write as they're very similar so that we only have to solve problems like this in one place.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

          Message: WT-2102 Combine the switch and force_write functions.
          All paths check whether they have closed the slot and need to retry
          creating a new slot if they get EBUSY.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/b833027ca90de4a3d0e64ec5761df22317913d87

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'} Message: WT-2102 Combine the switch and force_write functions. All paths check whether they have closed the slot and need to retry creating a new slot if they get EBUSY. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/b833027ca90de4a3d0e64ec5761df22317913d87
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2186 from wiredtiger/wt-2102

          WT-2102 Combine the switch and force_write functions.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/e1d6886824058b333495236b776b10fcd8fb74ae

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2186 from wiredtiger/wt-2102 WT-2102 Combine the switch and force_write functions. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/e1d6886824058b333495236b776b10fcd8fb74ae

            People

            • Assignee:
              sue.loverso Sue LoVerso
              Reporter:
              sue.loverso Sue LoVerso
              Participants:
              Last commenter:
              Michael Cahill
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 36 weeks, 4 days ago
                Date of 1st Reply: