[SERVER-5301] performance regression with many indexes and few results Created: 14/Mar/12  Updated: 11/Jul/16  Resolved: 22/Apr/12

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: 2.1.1

Type: Bug Priority: Major - P3
Reporter: Aaron Staple Assignee: Aaron Staple
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

Earlier in the 2.1 development cycle I made a change that requires us to generate all plans for many queries even if an index has been recorded for the relevant query pattern. (All the plans are generated, but only the recorded plan actually gets run.)

This resulted in a performance regession, which I have measured as follows:

In current code an optimal index query with nscanned==0 takes about 45% longer when there are 52 indexes than when there are 2 indexes. In 2.0, the corresponding penalty caused by the 50 additional indexes is 10%. (With increasing nscanned/nscannedObjects the total operation time increases and the measured (relative) performance difference diminishes.)

I think we can pretty easily restore the earlier performance by adding an additional metadata field that gets recorded along with the index for a query pattern.

The "simple id" mode is not affected by this regression.



 Comments   
Comment by auto [ 20/Apr/12 ]

Author:

{u'login': u'astaple', u'email': u'aaron@10gen.com', u'name': u'Aaron'}

Message: SERVER-5301 Store CandidatePlanCharacter in the query plan cache, and use its value to avoid pre populating query plans.
Branch: master
https://github.com/mongodb/mongo/commit/131dd557b9ea7d7088198c6558bb91d835294693

Comment by auto [ 20/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5301 Update indexp test to reflect query pattern cache flusing when an index becomes multikey.
Branch: master
https://github.com/mongodb/mongo/commit/40329ed29b27ba1854aef18b923eb0634a6d37ed

Comment by auto [ 20/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5301 Record optimal query plans in the query plan cache.
Branch: master
https://github.com/mongodb/mongo/commit/72e76dd5d8dc2a85ac1973e2bd00c5c2496c833c

Comment by auto [ 20/Apr/12 ]

Author:

{u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}

Message: SERVER-5301 Clear query plan cache when an index becomes multikey.
Branch: master
https://github.com/mongodb/mongo/commit/870e5cf7e25951c6dc8ed75e2d7343db4f0bfd8c

Generated at Thu Feb 08 03:08:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.