[SERVER-16492] thread gets stuck doing update for a long time Created: 10/Dec/14 Updated: 24/Jan/15 Resolved: 18/Dec/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 2.8.0-rc2 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Asya Kamsky | Assignee: | Alexander Gorrod |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
Testing YCSB 50/50 against zlib against replica set seems the easiest way to reproduce this scenario. While process is being made, one of the writing threads gets stuck on a single update for a long long time:
The stack traces (will attach) seem to indicate that they are compressing large pages. |
| Comments |
| Comment by Michael Cahill (Inactive) [ 18/Dec/14 ] | |||||||||||||||||||||||||||||||||||
|
duplicate of | |||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 17/Dec/14 ] | |||||||||||||||||||||||||||||||||||
|
I think this could be related or partly | |||||||||||||||||||||||||||||||||||
| Comment by Asya Kamsky [ 10/Dec/14 ] | |||||||||||||||||||||||||||||||||||
|
Here is the latest startup that reproduced at least one thread that ran for over 60 seconds in an update:
I started up new mongod with new db directory and populated it from another host using:
Then I ran the same workload file (it's 50/50) with equivalent of:
I ran db.currentOp() periodically and caught one that had been running for longer than 65 seconds (I did grab a full currentOp() output so if there was something in DB blocking it I presume it would be seen there. I've also got sar logs and mongostat for that time period, if it will help. | |||||||||||||||||||||||||||||||||||
| Comment by Daniel Pasette (Inactive) [ 10/Dec/14 ] | |||||||||||||||||||||||||||||||||||
|
asya, can you include mongod startup options, machine details (which aws instance and storage setup), and precise ycsb cmd for repro? |