[SERVER-26796] planCacheSetFilter doesn't work for queries emitted by pymongo to MongoDB 2.6 if projection is not empty Created: 26/Oct/16  Updated: 27/Oct/16  Resolved: 26/Oct/16

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Marcin Barczy?ski Assignee: Unassigned
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-13008 Encoding of projection for query plan... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

I know that MongoDB 2.6 is quite an old version, and the issue is not present in the newest MongoDB 3.2, but I can't switch to MongoDB 3.2 easily. I wonder if there is any workaround?
The following script exposes the problem:

#!/bin/bash
 
DB="test"
 
mongo <<__EOF
use $DB
db.e.drop()
db.e.insert({"key1": "some-val-1", "key2": "some-val-2"})
db.e.createIndex({'key1': 1})
db.e.createIndex({'key2': 1})
 
db.runCommand({"planCacheClearFilters": "e"})
db.runCommand({"planCacheSetFilter": "e", 
    "query": {"key1": 1, "key2": 1},
    "indexes": [ {"key1": 1} ]})
db.runCommand({"planCacheSetFilter": "e", 
    "query": {"key1": 1, "key2": 1},
    "projection": {"key1": 1},
    "indexes": [ {"key1": 1} ]})
db.runCommand({"planCacheListFilters": "e"})
__EOF
 
mongo <<__EOF
exp1 = db.e.find({"key1": "val1", "key2": "val2"}).explain()
exp2 = db.e.find({"key1": "val1", "key2": "val2"}, {"key1": 1}).explain()
print("MongoClient no projection: " + exp1.filterSet)
print("MongoClient projection: " + exp2.filterSet)
__EOF
 
python <<__EOF
from pymongo import MongoClient
 
cl = MongoClient()
db = cl["$DB"]
e = db['e']
 
curs = e.find({"key1": "val1", "key2": "val2"})
print "Python no projection:", curs.explain()['filterSet']
curs = e.find({"key1": "val1", "key2": "val2"}, projection=["key1"])
print "Python projection:", curs.explain()['filterSet']
__EOF



 Comments   
Comment by Marcin Barczy?ski [ 27/Oct/16 ]

behackett: Thank you for the workaround.

Comment by Bernie Hackett [ 26/Oct/16 ]

Duplicate of SERVER-13008

Comment by J Rassi [ 26/Oct/16 ]

behackett: the issue you describe above is fixed in SERVER-13008.

Comment by Bernie Hackett [ 26/Oct/16 ]

I can confirm that this is fixed in MongoDB 3.2, for both the find command and legacy OP_QUERY style queries.

Comment by Bernie Hackett [ 26/Oct/16 ]

The same problem exists in MongoDB 3.0.x. Weirdly, using True instead of 1 also does not fix the problem. You must use 1.0. This contradicts how projections normally work, and how they are documented:

https://docs.mongodb.com/v2.6/reference/method/db.collection.find/#definition

>>> pprint.pprint(db.e.find({"key1": "val1", "key2": "val2"}, projection={"key1": 1}).explain())
...
u'queryPlanner': {u'indexFilterSet': False,
                   u'namespace': u'test.e',
...
>>> pprint.pprint(db.e.find({"key1": "val1", "key2": "val2"}, projection={"key1": True}).explain())
 u'queryPlanner': {u'indexFilterSet': False,
                   u'namespace': u'test.e',
...
>>> pprint.pprint(db.e.find({"key1": "val1", "key2": "val2"}, projection={"key1": 1.0}).explain())
 u'queryPlanner': {u'indexFilterSet': True,
                   u'namespace': u'test.e',
...

Comment by Bernie Hackett [ 26/Oct/16 ]

This looks like a bug in the server. Trying the explain query in the shell the server logs this:

2016-10-26T12:54:00.254-0700 [conn7] Running query: query: { key1: "val1", key2: "val2" } sort: {} projection: { key1: 1.0 } skip: 0 limit: 0

With the python driver you get this:

2016-10-26T12:53:46.342-0700 [conn2] Running query: query: { key1: "val1", key2: "val2" } sort: {} projection: { key1: 1 } skip: 0 limit: 0

Note that the value for the projection is 1.0 with the shell (by default all numbers in javascript are doubles), but 1 with the python driver. Changing the python query to use 1.0 for the projection works around the problem:

>>> pprint.pprint(db.e.find({"key1": "val1", "key2": "val2"}, projection={"key1": 1.0}).explain())
{u'allPlans': [{u'cursor': u'BtreeCursor key1_1',
                u'indexBounds': {u'key1': [[u'val1', u'val1']]},
                u'indexOnly': False,
                u'isMultiKey': False,
                u'n': 0,
                u'nChunkSkips': 0,
                u'nscanned': 0,
                u'nscannedObjects': 0,
                u'scanAndOrder': False}],
 u'cursor': u'BtreeCursor key1_1',
 u'filterSet': True,
...

Generated at Thu Feb 08 04:13:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.