Critical - P2 Run the splitChunk command with out of bound splitKeys, the command fails. But it still updates the chunks, hence results in corrupted config database with overlap chunks and a chunk with reverse order.
We've tested 3.0.12, 3.2.8, 3.2.9-rc1, 3.3.10 and 3.3.11. All have the same issue.
The chunks before running the splitChunk command:
BEFORE: [
{
"_id" : "test.user-x_MinKey",
"lastmod" : Timestamp(1, 1),
"lastmodEpoch" : ObjectId("57b1229178e732ea6489534f"),
"ns" : "test.user",
"min" : {
"x" : { "$minKey" : 1 }
},
"max" : {
"x" : 0
},
"shard" : "shard0000"
},
{
"_id" : "test.user-x_0.0",
"lastmod" : Timestamp(1, 2),
"lastmodEpoch" : ObjectId("57b1229178e732ea6489534f"),
"ns" : "test.user",
"min" : {
"x" : 0
},
"max" : {
"x" : { "$maxKey" : 1 }
},
"shard" : "shard0000"
}
]
The chunks after running the splitChunk command:
AFTER: [
{
"_id" : "test.user-x_MinKey",
"lastmod" : Timestamp(1, 3),
"lastmodEpoch" : ObjectId("57b11ede5ed71a434707b87e"),
"ns" : "test.user",
"min" : {
"x" : { "$minKey" : 1 }
},
"max" : {
"x" : 2
},
"shard" : "shard0000"
},
{
"_id" : "test.user-x_0.0",
"lastmod" : Timestamp(1, 2),
"lastmodEpoch" : ObjectId("57b11ede5ed71a434707b87e"),
"ns" : "test.user",
"min" : {
"x" : 0
},
"max" : {
"x" : { "$maxKey" : 1 }
},
"shard" : "shard0000"
},
{
"_id" : "test.user-x_2.0",
"lastmod" : Timestamp(1, 4),
"lastmodEpoch" : ObjectId("57b11ede5ed71a434707b87e"),
"ns" : "test.user",
"min" : {
"x" : 2
},
"max" : {
"x" : 0
},
"shard" : "shard0000"
}
]
jstest attached. It fails with:
assert: [2] != [3] are not equal : Split chunks failed, but the chunks were updated in the config database doassert@src/mongo/shell/assert.js:15:14 assert.eq@src/mongo/shell/assert.js:51:5 @split_chunk_out_of_bound.js:36:1 2016-08-15T02:11:18.422+0000 E QUERY [thread1] Error: [2] != [3] are not equal : Split chunks failed, but the chunks were updated in the config database : doassert@src/mongo/shell/assert.js:15:14 assert.eq@src/mongo/shell/assert.js:51:5 @split_chunk_out_of_bound.js:36:1 failed to load: split_chunk_out_of_bound.js
Also, for 3.3.11 the primary shard fasserts (SERVER-24569) after config.chunks has been modified:
d20000| 2016-08-15T12:01:53.514+1000 I - [conn1] Fatal assertion 40221 IllegalOperation: cannot split chunk [{ x: MinKey }, { x: 0.0 }) at key { x: 2.0 } at src/mongo/db/s/split_chunk_command.cpp 436
d20000| 2016-08-15T12:01:53.514+1000 I - [conn1]
d20000|
d20000| ***aborting after fassert() failure
d20000|
d20000|
d20000| 2016-08-15T12:01:53.523+1000 F - [conn1] Got signal: 6 (Abort trap: 6).
fassert is not an apporpriate response to the server receiving a command with bad parameters. The command should just fail and return an error (with no side effects).
- is related to
-
SERVER-25630 Add validation of splitVector output
-
- Closed
-
- related to
-
SERVER-24569 Maintain the 'rangesToClean' and 'metadataInUse' sets on chunk migrations
-
- Closed
-