[SERVER-30937] Migration failed because of `already exists with different options` Created: 04/Sep/17  Updated: 15/Nov/21  Resolved: 01/Nov/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Zhang Youdong Assignee: Danny Hatcher (Inactive)
Resolution: Duplicate Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-769 Validate top-level & index spec field... Closed
duplicates SERVER-30344 prevent shards from implicitly creati... Closed
Operating System: ALL
Steps To Reproduce:

Create a shared cluster

Step1: enable sharding for test database

sh.enableSharding("test")

Step2: create index for some colletion

mongos> db.coll.createIndex({x: 1})
{
	"raw" : {
		"mymongo02/r101072137.sqa.zmf:9665" : {
			"createdCollectionAutomatically" : true,
			"numIndexesBefore" : 1,
			"numIndexesAfter" : 2,
			"ok" : 1,
			"$gleStats" : {
				"lastOpTime" : Timestamp(1504497721, 2),
				"electionId" : ObjectId("58eeece962cc998e3a94975d")
			}
		}
	},
	"ok" : 1
}

Shard1 (primary shard for test database)

db.coll.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "test.coll"
	},
	{
		"v" : 1,
		"key" : {
			"x" : 1
		},
		"name" : "x_1",
		"ns" : "test.coll"
	}
]

Shard2

db.coll.getIndexes()
[ ]

Step3: shardCollection for test.coll

sh.shardCollection("test.coll", {x: 1})
{ "collectionsharded" : "test.coll", "ok" : 1 }

Step4: create index again with different option

mongos> db.coll.createIndex({x: 1}, {safe: null})
{
	"raw" : {
		"mymongo02/r101072137.sqa.zmf:9665" : {
			"ok" : 0,
			"errmsg" : "Index with name: x_1 already exists with different options",
			"code" : 85
		},
		"mymongo03/r101072137.sqa.zmf:9666" : {
			"createdCollectionAutomatically" : true,
			"numIndexesBefore" : 1,
			"numIndexesAfter" : 2,
			"ok" : 1,
			"$gleStats" : {
				"lastOpTime" : Timestamp(1504497768, 2),
				"electionId" : ObjectId("59acc73aacadfa7926c9cabb")
			}
		}
	},
	"code" : 85,
	"ok" : 0,
	"errmsg" : "{ mymongo02/r101072137.sqa.zmf:9665: \"Index with name: x_1 already exists with different options\" }"
}

Shard1 (primary shard)

db.coll.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "test.coll"
	},
	{
		"v" : 1,
		"key" : {
			"x" : 1
		},
		"name" : "x_1",
		"ns" : "test.coll"
	}
]
mymongo02:PRIMARY>

Shard2

db.coll.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "test.coll"
	},
	{
		"v" : 1,
		"key" : {
			"x" : 1
		},
		"name" : "x_1",
		"ns" : "test.coll",
		"safe" : null
	}
]

Now two shards have the same index with different IndexOption, the migration will never success.

Participants:

 Description   

The migration in shared cluster stopped because of "transfer data error", after dive into the log and source code, I found that the migration dest failed when create index.

2017-09-04T10:43:04.485+0800 I SHARDING [migrateThread] about to log metadata event into changelog: { _id: "nu0b08400.cloud.nu16-2017-09-04T10:43:04.485+0800-59acbdb864a989db483e37a9", server: "nu0b08400.cloud.nu16", clientAddr: "", time: new Date(1504492984485), what: "moveChunk.to", ns: "node_club_dev.userscorenormals", details: { min: { _id: ObjectId('55f13740a33ff9d210931319') }, max: { _id: ObjectId('55f164bd2ab3c6000052e2ec') }, migrationSessionId: "d-2ze3b2adeeed3f84_d-2zef90f7483c0d74_59acbdb8ded555150c17945c", note: "aborted", errmsg: "failed to create index before migrating data.  error: IndexOptionsConflict: Index with pattern: { user_id: 1 } already exists with different options" } }

I have the three questions about this problem?

1. why invalid IndexOption can be specified when createIndex? f
2. why createIndex forward all the requests to shards without checking if index already exist in the primary shard?
3. why createIndex forward requests to all shards, even the shards doesn't hold the collection data?



 Comments   
Comment by Danny Hatcher (Inactive) [ 01/Nov/19 ]

1. why invalid IndexOption can be specified when createIndex?

This is addressed in SERVER-769 and fixed in the 3.4.0 release.

2. why createIndex forward all the requests to shards without checking if index already exist in the primary shard?
3. why createIndex forward requests to all shards, even the shards doesn't hold the collection data?

As of SERVER-30344 and the 3.6.0 release we only create the indexes if the shard has the collection.

I'm going to close this ticket as a duplicate.

Comment by conan [ 09/Nov/17 ]

this bug causes a online problem, looking forward

Comment by Zhang Youdong [ 08/Sep/17 ]

What's the progress of this issue? I noticed it's not assigned for few dayes.

Comment by Zhang Youdong [ 04/Sep/17 ]

Affect version: MongoDB 3.2.13

Generated at Thu Feb 08 04:25:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.