Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: WT2.6.1
    • Fix Version/s: WT2.7.0
    • Labels:
      None
    • # Replies:
      8
    • Last comment by Customer:
      true

      Description

      From SERVER-19293.

      When verify is run on a database, it should not panic. An error should be returned instead because MongoDB will then run salvage, which will fix the problem.

      2015-07-06T13:49:53.972-0500 E STORAGE  [initandlisten] WiredTiger (0) [1436208593:972487][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: read checksum error [4096B @ 9994240, 832067279 != 2155579310]
      2015-07-06T13:49:53.972-0500 E STORAGE  [initandlisten] WiredTiger (0) [1436208593:972512][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: collection-1595-7140502635356682714.wt: encountered an illegal file format or internal value
      ...
      2015-07-06T13:49:53.972-0500 E STORAGE  [initandlisten] WiredTiger (-31804) [1436208593:972525][16105:0x7fe889ce5bc0], file:collection-1595-7140502635356682714.wt, session.verify: the process must exit and restart: WT_PANIC: WiredTiger library panic
      ...
       mongod(__wt_panic+0x24) [0x13914e4]
       mongod(__wt_block_extlist_read+0x6E) [0x12e228e]
       mongod(__wt_block_extlist_read_avail+0x28) [0x12e2728]
       mongod(+0xEE54CF) [0x12e54cf]
       mongod(__wt_block_verify_start+0x108) [0x12e5648]
       mongod(__wt_verify+0x4AA) [0x1310afa]
      

        Issue Links

          Activity

          Hide
          michael.cahill Michael Cahill added a comment - - edited

          Keith Bostic, I've pushed a branch here for your review.

          I have also backported the change to 3.0 and kicked off a patch build here: https://evergreen.mongodb.com/patch/55a8b51c3ff122295800013f

          If you agree with the change, it should be possible for users hitting this problem to make progress using the binaries from the patch build. Check in with Ramon Fernandez about recommending that on SERVER-19293.

          Show
          michael.cahill Michael Cahill added a comment - - edited Keith Bostic , I've pushed a branch here for your review. I have also backported the change to 3.0 and kicked off a patch build here: https://evergreen.mongodb.com/patch/55a8b51c3ff122295800013f If you agree with the change, it should be possible for users hitting this problem to make progress using the binaries from the patch build. Check in with Ramon Fernandez about recommending that on SERVER-19293 .
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2002 If a file is corrupted, verify should not panic.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/b4e2e2b4be652e011fc3059ac37fdb21f303c824

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2002 If a file is corrupted, verify should not panic. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/b4e2e2b4be652e011fc3059ac37fdb21f303c824
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2002 Fix a typo: wrap if body in brackets.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/0c3f294e451766a6156245894ad8190a85195bd7

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2002 Fix a typo: wrap if body in brackets. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/0c3f294e451766a6156245894ad8190a85195bd7
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: WT-2002 Don't set block->verify until just before reading the extent list.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/d2a0318a2f6f97606228e0de44cba07d1113aa84

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: WT-2002 Don't set block->verify until just before reading the extent list. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/d2a0318a2f6f97606228e0de44cba07d1113aa84
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: Merge pull request #2070 from wiredtiger/verify-no-panic

          WT-2002 If a file is corrupted, verify should not panic.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/8af8b8a7ce21ca0ee6743c6af9313fd76817bc9c

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: Merge pull request #2070 from wiredtiger/verify-no-panic WT-2002 If a file is corrupted, verify should not panic. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/8af8b8a7ce21ca0ee6743c6af9313fd76817bc9c
          Hide
          michael.cahill Michael Cahill added a comment -

          I've contacted the customer in SERVER-19293. I believe the only thing left here is to backport this change to WiredTiger's mongodb-3.0 branch so it is included in the next 3.0 drop.

          Show
          michael.cahill Michael Cahill added a comment - I've contacted the customer in SERVER-19293 . I believe the only thing left here is to backport this change to WiredTiger's mongodb-3.0 branch so it is included in the next 3.0 drop.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'}

          Message: Merge pull request #2070 from wiredtiger/verify-no-panic

          WT-2002 If a file is corrupted, verify should not panic.
          (cherry picked from commit 8af8b8a7ce21ca0ee6743c6af9313fd76817bc9c)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/0969f76b03c0cfcad011c2b7cc2fa4548d65fc88

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith.bostic@mongodb.com'} Message: Merge pull request #2070 from wiredtiger/verify-no-panic WT-2002 If a file is corrupted, verify should not panic. (cherry picked from commit 8af8b8a7ce21ca0ee6743c6af9313fd76817bc9c) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/0969f76b03c0cfcad011c2b7cc2fa4548d65fc88
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}

          Message: WT-2002 MSVC compilers can't handle nested variadic macros. In other words, they mess up when there are nested uses of _VA_ARGS, merge the calls to WT_RET_MSG and WT_PANIC_RET in WT_BLOCK_ERROR so there's a single use of __VAR_ARGS_.

          Change the name of WT_BLOCK_ERROR to WT_BLOCK_RET to match WT_RET/WT_ERR naming.

          (cherry picked from commit c1c1ff43e95c750c7abc29fbd701314a9e23c26a)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/a6e130f1da240de9e6a39fe088c9b3ada72903af

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'} Message: WT-2002 MSVC compilers can't handle nested variadic macros. In other words, they mess up when there are nested uses of _ VA_ARGS , merge the calls to WT_RET_MSG and WT_PANIC_RET in WT_BLOCK_ERROR so there's a single use of __VAR_ARGS _. Change the name of WT_BLOCK_ERROR to WT_BLOCK_RET to match WT_RET/WT_ERR naming. (cherry picked from commit c1c1ff43e95c750c7abc29fbd701314a9e23c26a) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/a6e130f1da240de9e6a39fe088c9b3ada72903af

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 42 weeks, 2 days ago
                Date of 1st Reply: