[SERVER-13875] ensureIndex of 2dsphere index breaks after upgrading to 2.6 (with the new createIndex command) Created: 08/May/14  Updated: 19/Sep/15  Resolved: 15/May/15

Status: Closed
Project: Core Server
Component/s: Geo
Affects Version/s: 2.6.1
Fix Version/s: 2.6.11, 3.0.4, 3.1.3

Type: Bug Priority: Major - P3
Reporter: Derick Rethans Assignee: J Rassi
Resolution: Done Votes: 0
Labels: ET
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File server13875.js    
Issue Links:
Depends
Related
Backwards Compatibility: Minor Change
Operating System: ALL
Backport Completed:
Participants:

 Description   

On a collection, I have an existing 2dsphere index (created while still using MongoDB 2.4.x):

        {
                "v" : 1,
                "key" : {
                        "l" : "2dsphere"
                },
                "ns" : "demo.foursquare",
                "name" : "l_2dsphere"
        },

I have some code that adds stuff from foursquare, and this script runs PHP's ensureIndex() to create the 2dsphere index in case it doesn't exist. On the shell, the equivalant is:

db.foursquare.ensureIndex( { l: '2dsphere' } );

However, that generates:

{
        "ok" : 0,
        "errmsg" : "Index with name: l_2dsphere already exists with different options",
        "code" : 67
}

But I am not setting any new options.

However, when I create a 2dsphere index on a different field (loc instead of l):

db.foursquare.createIndex( { loc: '2dsphere' } );

Then it creates this index:

        {
                "v" : 1,
                "key" : {
                        "loc" : "2dsphere"
                },
                "name" : "loc_2dsphere",
                "ns" : "demo.foursquare",
                "2dsphereIndexVersion" : 2
        }

(Note the extra "2dsphereIndexVersion" : 2).

When I then run the createIndex again, I get (correctly):

{ "numIndexesBefore" : 4, "note" : "all indexes already exist", "ok" : 1 }

It looks like it breaks over the extra 2dsphereIndexVersion "option", but as user I have no influence over that not being set.



 Comments   
Comment by Githook User [ 19/May/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13875 Spec equivalence check ignore text/geo index versions

(cherry picked from commit 20e130aceda766c646d123d8a853ea5aabbeab26)
Branch: v2.6
https://github.com/mongodb/mongo/commit/137b088130d7f7fd4ebc8b09d79ed72de44925b5

Comment by Githook User [ 15/May/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13875 Spec equivalence check ignore text/geo index versions

(cherry picked from commit 20e130aceda766c646d123d8a853ea5aabbeab26)
Branch: v3.0
https://github.com/mongodb/mongo/commit/c5c6df8024cb1795172fdf7098e2ceacc3797c0d

Comment by Githook User [ 15/May/15 ]

Author:

{u'username': u'jrassi', u'name': u'Jason Rassi', u'email': u'rassi@10gen.com'}

Message: SERVER-13875 Spec equivalence check ignore text/geo index versions
Branch: master
https://github.com/mongodb/mongo/commit/20e130aceda766c646d123d8a853ea5aabbeab26

Comment by Alexander Komyagin [ 13/May/15 ]

rassi@10gen.com thanks for the clarification. I somehow missed it - sorry!

Comment by J Rassi [ 13/May/15 ]

alex.komyagin@10gen.com: that's scenario #5 from my list above.

Comment by Alexander Komyagin [ 13/May/15 ]

There is another scenario that was pointed out by one of our customers:

  1. Create a 2.4.x cluster with two shards, using replica sets
  2. Create a sharded collection with a 2dsphere index and distribute chunks between the shards
  3. Upgrade this cluster to 2.6.x.
  4. Wipe one of the replica set nodes and allow it to rebuild from the primary (to simulate adding or replacing a node). This new node will be created with an explicit v2 2dsphere index
  5. Make this rebuilt node primary for it's replica set
  6. Try moving chunks between shards

In this case, you will be completely unable to move any chunks between shards, but differently to the original case, this time the index versioning is now /explicitly/ wrong - i.e. the two shards now have different indexes and we now have a pile of data that has been "upgraded" to the V2 index. On a large dataset, this would take some sorting out...!

In other words, the problem might be deeper - we just shouldn't have implicit versions

Comment by J Rassi [ 13/May/15 ]

I propose changing the behavior of ensureIndex() to succeed as a no-op with "index already exists" if an index already exists on the same key pattern and differs only in index plugin version (2dsphereIndexVersion or textIndexVersion), rather than fail with "index options conflict". It seems to me that this will better conform to user expectations.

To illustrate, see how creating an index on a given 2dsphere key pattern 'kp' would work in the following scenarios (the behavior in scenarios #1/#3/#4 would remain unchanged, and the behavior in #2/#5 would change):

  1. ensureIndex(kp) where no index on 'kp' exists: create index as {2dsphereIndexVersion: 2}
  2. ensureIndex(kp) where index exists on 'kp' as {2dsphereIndexVersion: 1}: succeed with "index already exists" (previously would fail with "index options conflict")
  3. ensureIndex(kp) where index exists on 'kp' as {2dsphereIndexVersion: 2}: succeed with "index already exists"
  4. ensureIndex(kp, {2dsphereIndexVersion: x}) where index exists on 'kp' with same version: succeed with "index already exists"
  5. ensureIndex(kp, {2dsphereIndexVersion: x}) where index exists on 'kp' with different version: succeed with "index already exists" (previously would fail with "index options conflict")
Comment by Derick Rethans [ 08/May/14 ]

I am not specifying any other options that I already did, so at least the error message is wrong. I would also certainly not expect that my already running scripts would just stop working. The ensureIndex match should be done before the server adds any options of its own perhaps.

I also can not find this in http://docs.mongodb.org/master/release-notes/2.6-compatibility as an incompatibility.

Comment by Greg Studer [ 08/May/14 ]

This is basically WAD - the index defaults needed to change for new indexes. It may be trivial to make "2dsphereIndexVersion" : 1 in v2.6 and above basically unset the field completely, which is currently correctly handled. Keeping ticket open to check.

> It looks like it breaks over the extra 2dsphereIndexVersion "option", but as user I have no influence over that not being set.

The workaround is to reindex in v2.6 or, if you're testing during an upgrade, you can manually set '2dsphereIndexVersion' : 1 when creating the index in v2.4.

Generated at Thu Feb 08 03:33:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.