[SERVER-21690] Text Search - Performance Regression in 3.2.0 RC4 Created: 30/Nov/15 Updated: 03/Dec/15 Resolved: 01/Dec/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Text Search |
| Affects Version/s: | 3.2.0-rc4 |
| Fix Version/s: | 3.2.0-rc6 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Stuart Hall | Assignee: | J Rassi |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Steps To Reproduce: | See above |
||||||||||||
| Sprint: | QuInt D (12/14/15) | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
MongoDB 3.2.0 RC4 appears to have a substantial performance regression with full text searching Test Data
This data was then indexed using an "all fields" index:
This produces a test dataset of around 1.1GB with a text index of 155MB (measured with WT) Test process
The words used are as follows:
A simple test script ("testQuery_all.js") is attached to automate this process. Test results
Full results are available here: Source data is here:
|
| Comments |
| Comment by J Rassi [ 03/Dec/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
I really appreciate the report and your testing efforts. Thanks again, and keep posted for updates on | ||||||||||||||||||||||||||||||||||||||||
| Comment by Stuart Hall [ 03/Dec/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
Hi Jason, Here are the results of my testing. Just to be absolutely sure here, I've re-run all of my tests as my previous run were done on battery, so the CPU was being throttled which would have made these results incomparable with the previous set:
(Source data here: https://goo.gl/VJfR5p) Based on this, I would probably agree with what you said; that half of this problem is fixed and we're left with a specific degradation on phrase matching in the V3 text index, which is more apparent from the raw test timings. In order to confirm this, I tested 3.2.0-rc6 against the V2 text index format:
On the basis of this, I can confirm your findings and the fix for this half of the problem. I'll keep an eye on the other ticket ( Thanks, Stuart H. | ||||||||||||||||||||||||||||||||||||||||
| Comment by Stuart Hall [ 03/Dec/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
Hi Jason, Thanks for the update and the fix. I'll re-test against 3.2.0-rc6 this evening and will report back. SH | ||||||||||||||||||||||||||||||||||||||||
| Comment by J Rassi [ 03/Dec/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
I've identified two independent performance issues from the test results reported:
#1 above has been previously reported at #2 has not been reported previously, and has been fixed for 3.2.0-rc6 in this ticket under the above commit, and a regression test has been pushed to mongo-perf (ec26ca98). This is a regression introduced by Stuart, would you be willing to re-run your tests against 3.2.0-rc6? | ||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 01/Dec/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
Author: {u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}Message: | ||||||||||||||||||||||||||||||||||||||||
| Comment by Stuart Hall [ 30/Nov/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
No problem at all. I'll keep an eye out for an updated build with a fix and will re-test. SH | ||||||||||||||||||||||||||||||||||||||||
| Comment by Martin Bligh [ 30/Nov/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
stuart.hall@masternaut.com Thanks very much for the very clear and easy to reproduce bug report. We think we understand the cause of this now, and will be addressing very soon. | ||||||||||||||||||||||||||||||||||||||||
| Comment by Stuart Hall [ 30/Nov/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
Hi Ramon, I wiped the DB data folder and performed the following process:
I then stopped the mongod and have attached the single diagnostic file (metrics.2015-11-30T16-53-52Z-00000) to the URL above. Results were quicker (393,775ms) than before, but this is probably because my laptop is powered rather than running on battery, so will run the CPU quicker. Either way, they are still 75x slower than 2.6.11 Thanks, Stuart H. | ||||||||||||||||||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 30/Nov/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
stuart.hall@masternaut.com, can you please compress the contents of <dbpath>/diagnostic.data and upload them here? Thanks, | ||||||||||||||||||||||||||||||||||||||||
| Comment by Stuart Hall [ 30/Nov/15 ] | ||||||||||||||||||||||||||||||||||||||||
|
One detail I forgot to add - in each case, I imported the data into a clean instance of mongod and created the index directly on the data. The textIndexVersion was therefore the default used by that given combination of engine and version. (i.e. version 2 for 2.6.11 and 3.0.7 and version 3 for 3.2.0 RC4). I've not done any testing of mixing engine and index versions at this time. |