[SERVER-21887] $sample takes disproportionately long time on newly created collection Created: 14/Dec/15 Updated: 23/Nov/16 Resolved: 21/Dec/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework, WiredTiger |
| Affects Version/s: | 3.2.0 |
| Fix Version/s: | 3.2.3, 3.3.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Asya Kamsky | Assignee: | Keith Bostic (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | code-only | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Operating System: | Linux | ||||||||||||||||||||||||
| Backport Completed: | |||||||||||||||||||||||||
| Steps To Reproduce: |
|
||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Description |
|
Even though This only happens to collection that was just created. Restarting mongod process "fixes" this symptom. |
| Comments |
| Comment by Githook User [ 29/Jan/16 ] |
|
Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}Message: Import wiredtiger-wiredtiger-2.7.0-559-g07966a4.tar.gz from wiredtiger branch mongodb-3.2 ref: 3c2ad56..07966a4
|
| Comment by Keith Bostic (Inactive) [ 21/Dec/15 ] |
|
The issues in the randomness of searches in skip-list only trees are resolved by |
| Comment by Keith Bostic (Inactive) [ 18/Dec/15 ] |
|
The search itself is now fast enough, but there are still issues in the randomness of searches in skip-list only trees. |
| Comment by Githook User [ 16/Dec/15 ] |
|
Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}Message: Merge pull request #2394 from wiredtiger/
|
| Comment by Githook User [ 16/Dec/15 ] |
|
Author: {u'username': u'keithbostic', u'name': u'Keith Bostic', u'email': u'keith@wiredtiger.com'}Message: |
| Comment by Alexander Gorrod [ 15/Dec/15 ] |
|
Thanks asya That does indeed look like the likely cause. I have talked to keith.bostic about the likely cause, and we have a plan of attack. Thanks for the reproducer and detailed information. |
| Comment by Alexander Gorrod [ 14/Dec/15 ] |
|
cc wiredtiger-escalation We need to get to the bottom of this one. |
| Comment by Lucas Hrabovsky (Inactive) [ 14/Dec/15 ] |
|
Any way to detect that a collection is a "new collection"? If so, it would be trivial to make compass use client-side reservoir on the fly by just patching in 1 place and including in weekly patch release: https://github.com/mongodb-js/collection-sample cc compass-escalation in case this pops up in CS tickets |
| Comment by Asya Kamsky [ 14/Dec/15 ] |
|
I've confirmed that this happens to newly created (inserted into) large collections only - restarting mongod always "clears" the problem. |
| Comment by Charlie Swanson [ 14/Dec/15 ] |
|
I've attached a reproduction script, and assigned to michael.cahill. Let me know if there's anything else I can help with! |
| Comment by Charlie Swanson [ 14/Dec/15 ] |
|
Attached a nice graph of where the time is going. It seems like we're stuck in __wt_row_random. This confirms it's internal to WiredTiger. I'll get a nice reproduction and pass the information along to the WiredTiger folks. |
| Comment by Asya Kamsky [ 14/Dec/15 ] |
|
Testing whether only newly created collection displays this symptom (will restart server then to see if that resolves it again). |
| Comment by Asya Kamsky [ 14/Dec/15 ] |
|
3.2.0 |
| Comment by Charlie Swanson [ 14/Dec/15 ] |
|
What version of mongod is this? |