Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-30937

Migration failed because of `already exists with different options`

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Sharding
    • Labels:
      None
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      Create a shared cluster

      Step1: enable sharding for test database

      sh.enableSharding("test")
      

      Step2: create index for some colletion

      mongos> db.coll.createIndex({x: 1})
      {
      	"raw" : {
      		"mymongo02/r101072137.sqa.zmf:9665" : {
      			"createdCollectionAutomatically" : true,
      			"numIndexesBefore" : 1,
      			"numIndexesAfter" : 2,
      			"ok" : 1,
      			"$gleStats" : {
      				"lastOpTime" : Timestamp(1504497721, 2),
      				"electionId" : ObjectId("58eeece962cc998e3a94975d")
      			}
      		}
      	},
      	"ok" : 1
      }
      

      Shard1 (primary shard for test database)

      db.coll.getIndexes()
      [
      	{
      		"v" : 1,
      		"key" : {
      			"_id" : 1
      		},
      		"name" : "_id_",
      		"ns" : "test.coll"
      	},
      	{
      		"v" : 1,
      		"key" : {
      			"x" : 1
      		},
      		"name" : "x_1",
      		"ns" : "test.coll"
      	}
      ]
      

      Shard2

      db.coll.getIndexes()
      [ ]
      

      Step3: shardCollection for test.coll

      sh.shardCollection("test.coll", {x: 1})
      { "collectionsharded" : "test.coll", "ok" : 1 }
      

      Step4: create index again with different option

      mongos> db.coll.createIndex({x: 1}, {safe: null})
      {
      	"raw" : {
      		"mymongo02/r101072137.sqa.zmf:9665" : {
      			"ok" : 0,
      			"errmsg" : "Index with name: x_1 already exists with different options",
      			"code" : 85
      		},
      		"mymongo03/r101072137.sqa.zmf:9666" : {
      			"createdCollectionAutomatically" : true,
      			"numIndexesBefore" : 1,
      			"numIndexesAfter" : 2,
      			"ok" : 1,
      			"$gleStats" : {
      				"lastOpTime" : Timestamp(1504497768, 2),
      				"electionId" : ObjectId("59acc73aacadfa7926c9cabb")
      			}
      		}
      	},
      	"code" : 85,
      	"ok" : 0,
      	"errmsg" : "{ mymongo02/r101072137.sqa.zmf:9665: \"Index with name: x_1 already exists with different options\" }"
      }
      

      Shard1 (primary shard)

      db.coll.getIndexes()
      [
      	{
      		"v" : 1,
      		"key" : {
      			"_id" : 1
      		},
      		"name" : "_id_",
      		"ns" : "test.coll"
      	},
      	{
      		"v" : 1,
      		"key" : {
      			"x" : 1
      		},
      		"name" : "x_1",
      		"ns" : "test.coll"
      	}
      ]
      mymongo02:PRIMARY>
      

      Shard2

      db.coll.getIndexes()
      [
      	{
      		"v" : 1,
      		"key" : {
      			"_id" : 1
      		},
      		"name" : "_id_",
      		"ns" : "test.coll"
      	},
      	{
      		"v" : 1,
      		"key" : {
      			"x" : 1
      		},
      		"name" : "x_1",
      		"ns" : "test.coll",
      		"safe" : null
      	}
      ]
      

      Now two shards have the same index with different IndexOption, the migration will never success.

      Show
      Create a shared cluster Step1: enable sharding for test database sh.enableSharding( "test" ) Step2: create index for some colletion mongos> db.coll.createIndex({x: 1 }) { "raw" : { "mymongo02/r101072137.sqa.zmf:9665" : { "createdCollectionAutomatically" : true , "numIndexesBefore" : 1 , "numIndexesAfter" : 2 , "ok" : 1 , "$gleStats" : { "lastOpTime" : Timestamp( 1504497721 , 2 ), "electionId" : ObjectId( "58eeece962cc998e3a94975d" ) } } }, "ok" : 1 } Shard1 (primary shard for test database) db.coll.getIndexes() [ { "v" : 1 , "key" : { "_id" : 1 }, "name" : "_id_" , "ns" : "test.coll" }, { "v" : 1 , "key" : { "x" : 1 }, "name" : "x_1" , "ns" : "test.coll" } ] Shard2 db.coll.getIndexes() [ ] Step3: shardCollection for test.coll sh.shardCollection( "test.coll" , {x: 1 }) { "collectionsharded" : "test.coll" , "ok" : 1 } Step4: create index again with different option mongos> db.coll.createIndex({x: 1 }, {safe: null }) { "raw" : { "mymongo02/r101072137.sqa.zmf:9665" : { "ok" : 0 , "errmsg" : "Index with name: x_1 already exists with different options" , "code" : 85 }, "mymongo03/r101072137.sqa.zmf:9666" : { "createdCollectionAutomatically" : true , "numIndexesBefore" : 1 , "numIndexesAfter" : 2 , "ok" : 1 , "$gleStats" : { "lastOpTime" : Timestamp( 1504497768 , 2 ), "electionId" : ObjectId( "59acc73aacadfa7926c9cabb" ) } } }, "code" : 85 , "ok" : 0 , "errmsg" : "{ mymongo02/r101072137.sqa.zmf:9665: \"Index with name: x_1 already exists with different options\" }" } Shard1 (primary shard) db.coll.getIndexes() [ { "v" : 1 , "key" : { "_id" : 1 }, "name" : "_id_" , "ns" : "test.coll" }, { "v" : 1 , "key" : { "x" : 1 }, "name" : "x_1" , "ns" : "test.coll" } ] mymongo02:PRIMARY> Shard2 db.coll.getIndexes() [ { "v" : 1 , "key" : { "_id" : 1 }, "name" : "_id_" , "ns" : "test.coll" }, { "v" : 1 , "key" : { "x" : 1 }, "name" : "x_1" , "ns" : "test.coll" , "safe" : null } ] Now two shards have the same index with different IndexOption, the migration will never success.

      Description

      The migration in shared cluster stopped because of "transfer data error", after dive into the log and source code, I found that the migration dest failed when create index.

      2017-09-04T10:43:04.485+0800 I SHARDING [migrateThread] about to log metadata event into changelog: { _id: "nu0b08400.cloud.nu16-2017-09-04T10:43:04.485+0800-59acbdb864a989db483e37a9", server: "nu0b08400.cloud.nu16", clientAddr: "", time: new Date(1504492984485), what: "moveChunk.to", ns: "node_club_dev.userscorenormals", details: { min: { _id: ObjectId('55f13740a33ff9d210931319') }, max: { _id: ObjectId('55f164bd2ab3c6000052e2ec') }, migrationSessionId: "d-2ze3b2adeeed3f84_d-2zef90f7483c0d74_59acbdb8ded555150c17945c", note: "aborted", errmsg: "failed to create index before migrating data.  error: IndexOptionsConflict: Index with pattern: { user_id: 1 } already exists with different options" } }
      

      I have the three questions about this problem?

      1. why invalid IndexOption can be specified when createIndex? f
      2. why createIndex forward all the requests to shards without checking if index already exist in the primary shard?
      3. why createIndex forward requests to all shards, even the shards doesn't hold the collection data?

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              daniel.hatcher Daniel Hatcher (Inactive)
              Reporter:
              zyd_com Youdong Zhang
              Participants:
              Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: