[SERVER-13769] Distinct command on indexed field with geo predicate fails to execute Created: 28/Apr/14  Updated: 11/Jul/16  Resolved: 01/May/14

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.6.0
Fix Version/s: 2.6.2, 2.7.0

Type: Bug Priority: Major - P3
Reporter: Lee Huffman Assignee: J Rassi
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Operating System: ALL
Backport Completed:
Participants:

 Description   
Issue Status as of June 6, 2014

ISSUE SUMMARY
The distinct command when run on an indexed field fails when the query portion of the command is on a field with a geo index (2d, 2dsphere).

Example:

db.foo.ensureIndex({a:1})
db.foo.ensureIndex({"loc":"2d"})
db.foo.distinct("a", {loc: {$nearSphere: [0,0]}})

This will trigger the bug because a is indexed and the query on loc uses the geo index.

USER IMPACT
The distinct command fails. A a stack trace is included in the log file.

WORKAROUNDS
A workaround is to use the aggregation framework for the distinct calculation. For above example on a 2d index, the equivalent aggregation call would be:

db.foo.aggregate( { $geoNear: { near: [0, 0], distanceField: "dist" } }, 
                  { $group: {_id: "$a" } } )
{ "_id" : 1 }
{ "_id" : 4 }
{ "_id" : 2 }
{ "_id" : 5 }
{ "_id" : 3 }

Note that the aggregation call returns a cursor and the format differs slightly from the distinct format.

AFFECTED VERSIONS
Versions 2.6.0 and 2.6.1 are affected by this issue.

FIX VERSION
The fix is included in the 2.6.2 production release.

RESOLUTION DETAILS
The query planner now clears the index tags in the case of geo indexes.

Original description

In a collection that has the following _id:

_id: {id: 'xxxx', d: MongoDate(xxx)}

Calling a distinct query with the key of "_id.id" causes a failure in mongo as seen below.

This error occurs in 2.6.0 and 2.6.1rc0. Mongodb 2.4.9 or earlier, does not have this error.

If I copy the _id.id to just a "id" then run distinct on "id" the query runs properly.

query:

db.dailyArchive201404->distinct("_id.id", { ... });

Error:

2014-04-27T18:46:23.438+0000 [conn18] dailyArchive201404 0x11c1421 0x1163699 0x11474de 0xd7c6e6 0xd7c3d3 0xd837f8 0xd1b7c6 0xd1d832 0xd1f7e6 0xd20873 0x9344b5 0xa1e92a 0xa1f89e 0xa21156 0xd4dbb7 0xb973f2 0xb999d2 0x76b77f 0x117779b 0x7ffae5eedf18 
 /opt/mongodb/current/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11c1421]
 /opt/mongodb/current/bin/mongod(_ZN5mongo10logContextEPKc+0x159) [0x1163699]
 /opt/mongodb/current/bin/mongod(_ZN5mongo12verifyFailedEPKcS1_j+0x17e) [0x11474de]
 /opt/mongodb/current/bin/mongod(_ZN5mongo20QueryPlannerIXSelect11rateIndicesEPNS_15MatchExpressionESsRKSt6vectorINS_10IndexEntryESaIS4_EE+0x916) [0xd7c6e6]
 /opt/mongodb/current/bin/mongod(_ZN5mongo20QueryPlannerIXSelect11rateIndicesEPNS_15MatchExpressionESsRKSt6vectorINS_10IndexEntryESaIS4_EE+0x603) [0xd7c3d3]
 /opt/mongodb/current/bin/mongod(_ZN5mongo12QueryPlanner4planERKNS_14CanonicalQueryERKNS_18QueryPlannerParamsEPSt6vectorIPNS_13QuerySolutionESaIS9_EE+0x1c38) [0xd837f8]
 /opt/mongodb/current/bin/mongod(_ZN5mongo19getRunnerAlwaysPlanEPNS_10CollectionEPNS_14CanonicalQueryERKNS_18QueryPlannerParamsEPPNS_6RunnerE+0x86) [0xd1b7c6]
 /opt/mongodb/current/bin/mongod(_ZN5mongo9getRunnerEPNS_10CollectionEPNS_14CanonicalQueryEPPNS_6RunnerEm+0x932) [0xd1d832]
 /opt/mongodb/current/bin/mongod(_ZN5mongo9getRunnerEPNS_14CanonicalQueryEPPNS_6RunnerEm+0x96) [0xd1f7e6]
 /opt/mongodb/current/bin/mongod(_ZN5mongo17getRunnerDistinctEPNS_10CollectionERKNS_7BSONObjERKSsPPNS_6RunnerE+0x1003) [0xd20873]
 /opt/mongodb/current/bin/mongod(_ZN5mongo15DistinctCommand3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x915) [0x9344b5]
 /opt/mongodb/current/bin/mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa1e92a]
 /opt/mongodb/current/bin/mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xc5e) [0xa1f89e]
 /opt/mongodb/current/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c6) [0xa21156]
 /opt/mongodb/current/bin/mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x2307) [0xd4dbb7]
 /opt/mongodb/current/bin/mongod() [0xb973f2]
 /opt/mongodb/current/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x442) [0xb999d2]
 /opt/mongodb/current/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x9f) [0x76b77f]
 /opt/mongodb/current/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4fb) [0x117779b]
 /lib64/libpthread.so.0(+0x7f18) [0x7ffae5eedf18]



 Comments   
Comment by Githook User [ 15/May/14 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13769 QueryPlanner::plan() has to clear tags from query tree
(cherry picked from commit 8d48e785906d31dda3107a4a5a74ef940b4f2c89)
Branch: v2.6
https://github.com/mongodb/mongo/commit/28bab1646931c346ff2e0fbc036ae4935d8c31b8

Comment by Githook User [ 15/May/14 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13769 geo_distinct.js: rewrite, add test cases
(cherry picked from commit 066ee29fd28a32b539d98737d2a20f8f9a0c5497)
Branch: v2.6
https://github.com/mongodb/mongo/commit/8ff445004bda16b05fcdc15d9276578e73e5f66d

Comment by Githook User [ 01/May/14 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13769 QueryPlanner::plan() has to clear tags from query tree
Branch: master
https://github.com/mongodb/mongo/commit/8d48e785906d31dda3107a4a5a74ef940b4f2c89

Comment by Githook User [ 01/May/14 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13769 geo_distinct.js: rewrite, add test cases
Branch: master
https://github.com/mongodb/mongo/commit/066ee29fd28a32b539d98737d2a20f8f9a0c5497

Comment by J Rassi [ 28/Apr/14 ]

Thanks for the additional info. I am now able to reproduce this as follows:

db.foo.ensureIndex({a:1})
db.foo.ensureIndex({"loc":"2d"})
db.foo.distinct("a", {loc: {$nearSphere: [0,0]}})

Please watch this ticket for updates on when a fix will be available.

Comment by Lee Huffman [ 28/Apr/14 ]

1)

db.dailyArchive201404->distinct("_id.id", 
{
    _id.d : MongoDate(1398556800),
    loc: {
        $nearSphere: [-80.40894,37.12985],
        $maxDistance: 0.012630230733009
    }
}
);

2) Indexes:

[
        {
                "v" : 1,
                "key" : {
                        "_id" : 1
                },
                "ns" : "dailyArchive201404",
                "name" : "_id_"
        },
        {
                "v" : 1,
                "key" : {
                        "_id.id" : 1
                },
                "ns" : "dailyArchive201404",
                "name" : "_id_id_1",
                "dropDups" : false,
                "background" : false
        },
        {
                "v" : 1,
                "key" : {
                        "loc" : "2d",
                        "_id.d" : 1
                },
                "ns" : "dailyArchive201404",
                "name" : "loc_2d__id_d_1",
                "dropDups" : false,
                "background" : false
        }
]

Note, in testing 2.6, if I copy the _id.id to a new "id" field, then run the distinct query with key="id" instead of "_id.id" then the query works in 2.6

Comment by J Rassi [ 28/Apr/14 ]

Hi,

Thanks for reporting this problem. I'd like to gather additional information to help further diagnose the issue.

1. You noted that the query that you ran that reproduced the failure was the following:

db.dailyArchive201404->distinct("_id.id", { ... });

Could you please provide what you replaced "..." above with when reproducing this failure?

2. Could you please provide the output of running the following at the mongo shell:

db.dailyArchive201404.getIndexes()

Thanks.

~ Jason Rassi

Generated at Thu Feb 08 03:32:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.