[SERVER-22199] Collection drop command during checkpoint causes complete stall until end of checkpoint Created: 15/Jan/16 Updated: 18/Nov/16 Resolved: 02/Feb/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage, WiredTiger |
| Affects Version/s: | None |
| Fix Version/s: | 3.2.3, 3.3.2 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bruce Lucas (Inactive) | Assignee: | Michael Cahill (Inactive) |
| Resolution: | Done | Votes: | 1 |
| Labels: | WTplaybook, code-only | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Completed: | |||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
Here's an example:
Stack traces collected during the stall show why: multiple operations are all stuck in dropAllQueued which in turn is blocked in __wt_session_drop:
The FTDC thread is similarly stuck:
I believe the sequence is:
It looks like there are two problems:
|
| Comments |
| Comment by Michael Cahill (Inactive) [ 22/Feb/16 ] |
|
michael@aorato.com, yes, this issue is fixed in 3.2.3. No changes are required to the application: drops during checkpoints will now return immediately without blocking. The disk space is reclaimed asynchronously. |
| Comment by Michael Dolinsky [ 21/Feb/16 ] |
|
Just to make sure |
| Comment by Ramon Fernandez Marina [ 02/Feb/16 ] |
|
That is correct michael@aorato.com. We're working on a 3.2.2-rc0 release candidate, scheduled for the coming days, and it would be useful if you could test it and confirm that the fix produces the desired results in your use case. |
| Comment by Michael Dolinsky [ 02/Feb/16 ] |
|
Thanks. |
| Comment by Githook User [ 02/Feb/16 ] |
|
Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: (cherry picked from commit aa98ee551e0fcface0a1c9d63001c269c068d9ac) |
| Comment by Githook User [ 02/Feb/16 ] |
|
Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}Message: |
| Comment by Ramon Fernandez Marina [ 21/Jan/16 ] |
|
michael@aorato.com, a short-term fix ( |
| Comment by Michael Dolinsky [ 21/Jan/16 ] |
|
This is a critical fix for us as we are creating and dropping collections on an hourly basis and due to the stall we could lose data that would otherwise needs to be inserted. Thank you. |