[SERVER-5715] Combinatorial limit of $in affects advanced querying Created: 26/Apr/12 Updated: 06/Dec/22 Resolved: 16/Aug/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | 2.0.4 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Peter Lundberg | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Done | Votes: | 3 |
| Labels: | query-44-grooming | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query
|
| Operating System: | ALL |
| Participants: |
| Description |
|
We have a case of searching for changed content as simplified in the example below. If however a 'power user' chooses many groups/contexts to participate in and has advanced filtering on prefered tags we get "Combinatorial limit of $in partitioning of result set exceeded". Could this not be optimized differently or at least have a fallback stratagy (filtering the result of the index/shard lookup for the least selective of the $in conditions? Or do you have some other suggestions? Perhaps related to Example query that fails with {code : 13385}. This is simplified and does not show limit and ordering. refered fields are ensured index in the correct order to normally work fast. We do not shard. The application has CMS type data and the search applies the user's graned roles, subscribed contexts and tags: db.content.find( { grant_read_ids: { '$in': [ "ALL", "1", "118189", "7", "161990", "162060", "162068", "166651", "166917", "161988", "167482", "167493", "167734", "166946", "167253", "167979", "167696", "167981", "167255", "168647", "168648", "166841", "168835", "168837", "170147", "170434", "169597", "170805", "171277", "170485", "171631", "171556", "166942", "176623", "171320", "180642", "181904", "181943", "182273", "182777", "182937", "184197", "184196", "184547", "159112", "155276", "161246", "157318", "160939", "153486", "157272", "166267", "158810", "155087", "154227", "160254", "156399", "168140", "155647", "158005", "161454", "156569", "159250", "153454", "157032", "153295", "154226", "157098", "155570", "154933", "160553", "158639", "154222", "159752", "159162", "155667", "161232", "157209", "158637", "159766", "158531", "154378", "154574", "167851" ] }, , , } ).count() |
| Comments |
| Comment by David Storch [ 16/Aug/19 ] | |||||
|
In recent versions (at least versions since 2.6, to my knowledge), $in queries will not fail due to reaching a combinatorial limit. I believe this issue has been addressed by refactors to the query planning code. Closing as "Gone Away". | |||||
| Comment by siddharth.singh@10gen.com [ 14/Dec/12 ] | |||||
|
Hi George, This ticket is not scheduled with a release yet. Unfortunately, I cannot predict when this might get tied to a release either. We take a number of factors including ,but not limited to, the number of up votes and the community feedback to prioritize on server tickets. I suggest that you up vote on the ticket and add yourself to the watch list to receive notifications when this ticket gets scheduled. Thanks. | |||||
| Comment by George P. [ 14/Dec/12 ] | |||||
|
Hi Siddharth, is there any update on this? | |||||
| Comment by Peter Lundberg [ 28/Apr/12 ] | |||||
|
Ok, a workaround if others get into this problem is to use a regex. Note this affects performance quite a bit. Eg
| |||||
| Comment by siddharth.singh@10gen.com [ 26/Apr/12 ] | |||||
|
We are looking into options around suitably increasing these limits and other optimizations/fallback strategy but its not scheduled yet. |