[SERVER-18460] Segfault during eviction under load Created: 13/May/15 Updated: 09/Jul/15 Resolved: 18/May/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage, WiredTiger |
| Affects Version/s: | 3.1.2 |
| Fix Version/s: | 3.0.4, 3.1.3 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Michael Grundy | Assignee: | Michael Cahill (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | 32qa, FT, next-drop | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Completed: | |||||||||||||||||||||
| Steps To Reproduce: | mongod --storageEngine=wiredTiger run mongo-perf in loop until it crashes using the attached test case.
Get mongo-perf from https://github.com/mongodb/mongo-perf, put the crashwt.js in the testcases subdirectory. |
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
Server segfaults during the eviction process. This stack trace is with
|
| Comments |
| Comment by Githook User [ 10/Jun/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: Disable eviction in a tree while it is being marked dead. refs Conflicts: | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Michael Cahill (Inactive) [ 26/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Cherry picked here: https://github.com/wiredtiger/wiredtiger/commit/a1ec3b331a83b84b716ed3477d1f1fe3e4d6fdfd | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Michael Cahill (Inactive) [ 18/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Should be resolved by https://evergreen.mongodb.com/version/mongodb_mongo_master_4f0e70b66182cbb872c4e5eefda23f1c58bdaab7 | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 15/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: Disable eviction in a tree while it is being marked dead. refs | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 15/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: Never evict during an LRU walk, including when clearing the walk point. Found via a stack trace in the referenced issue, but not the cause of the crash. refs | ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Michael Grundy [ 14/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
On the recent master version of mongod, there is additional information prior to the backtrace
If you look at the code at the segfault address, you'll see that bm is obviously null (bm is copied to %rdi as it's also the first arg):
| ||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Michael Grundy [ 13/May/15 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||
|
Also able to repro on db version: 3.1.3-pre- The problem looks like it happens in __rec_split_init at mongo/src/third_party/wiredtiger/src/reconcile/rec_write.c:1635 Here's an analyzed backtrace from recent master:
|