[SERVER-31905] in 3.6, __wt_update_obsolete_check, uses significant cpu Created: 10/Nov/17 Updated: 12/Dec/17 Resolved: 21/Nov/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | WiredTiger |
| Affects Version/s: | 3.6.0-rc3 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Jim Van Fleet | Assignee: | Keith Bostic (Inactive) |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||
| Issue Links: |
|
||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Steps To Reproduce: | This regularly occurs after a few minutes of running the ycsb benchmark. Happens on both x86 and power |
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
the calls to __wt_update_obsolete_check from __col_instantiate and __row_instantiate were not in 3.4. between 3.4 and 3.6-r2 these calls were added and, as a result, __wt_update_obsolete_check cpu usage builds to over 20% of system |
| Comments |
| Comment by Keith Bostic (Inactive) [ 21/Nov/17 ] | |||||||||||||||||||||
|
Fixed by | |||||||||||||||||||||
| Comment by Keith Bostic (Inactive) [ 21/Nov/17 ] | |||||||||||||||||||||
|
Thank you, jvf! I'll go ahead and close this report then, please don't hesitate to re-open it if there are any follow-on issues. | |||||||||||||||||||||
| Comment by Jim Van Fleet [ 21/Nov/17 ] | |||||||||||||||||||||
|
Keith – I just tried this fix – I think it takes care of the problem that I reported. I want to do some more testing and will finish up tomorrow – nice job! | |||||||||||||||||||||
| Comment by Keith Bostic (Inactive) [ 20/Nov/17 ] | |||||||||||||||||||||
|
jvf, there's a change we put into place last night that I believe may fix this problem, would you be able to test if it fixes the problem you're seeing? I've attached the diff to this ticket (ba357b7.diff), and the patch would be applied in the directory mongo/src/third_party/wiredtiger/src/btree/, to the file __row_modify.c. | |||||||||||||||||||||
| Comment by Jim Van Fleet [ 15/Nov/17 ] | |||||||||||||||||||||
|
I slightly lied - we're using workload "mixed" not workloada (they are I included another snippet of perf top, the start-script, the ycsb the script is generally started something like sh tesCursor.sh localhost 27017 20 3_6fix Jim | |||||||||||||||||||||
| Comment by Keith Bostic (Inactive) [ 15/Nov/17 ] | |||||||||||||||||||||
|
jvf, would you please post a copy of the workloada that you're running, and any command-line arguments? Thanks! | |||||||||||||||||||||
| Comment by Jim Van Fleet [ 15/Nov/17 ] | |||||||||||||||||||||
|
Hi Keith – My bad – what I sent was a snippet from perf top and somehow I put My appolgies Jim | |||||||||||||||||||||
| Comment by Keith Bostic (Inactive) [ 15/Nov/17 ] | |||||||||||||||||||||
|
Hi, jvf. You said:
But I'm not seeing __row_instantiate or __wt_update_obsolete_check in the perf report? I'm sure there's something obvious I'm missing – it makes sense to me that if the lookaside file is getting involved they could be there, but I don't see them? alexander.gorrod, this is a zipfian distribution with modify, so yes, I suspect it's similar to | |||||||||||||||||||||
| Comment by Alexander Gorrod [ 15/Nov/17 ] | |||||||||||||||||||||
|
keith.bostic I've assigned this ticket to you because I believe it may be a different manifestation of the issue outlined in | |||||||||||||||||||||
| Comment by Jim Van Fleet [ 13/Nov/17 ] | |||||||||||||||||||||
|
[attachment "diag_log_3_6rc3.tar.gz" deleted by Jim Van | |||||||||||||||||||||
| Comment by Jim Van Fleet [ 13/Nov/17 ] | |||||||||||||||||||||
|
Sorry, used 3.6.0-rc3 – what I have seen is similar behavior in ycsb workloada – 50% read, 50% write FYI – throughput for rc3 is 163173; for 3.4 is 300110 perf top output – rc3
perf top output – 3.4
| |||||||||||||||||||||
| Comment by Alexander Gorrod [ 13/Nov/17 ] | |||||||||||||||||||||
|
jvf Thanks for trying out the release candidate and sending feedback. Could you please:
|