-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: 6.0.0, 7.0.0, 8.0.0-rc0
-
Component/s: None
-
Labels:
-
Catalog and Routing
-
ALL
-
3
Collection creation in transactions involving more than one shard is disallowed because there is no conflict detection machinery at prepare time for distributed transactions (SERVER-46105 should address that).
Write operations on encrypted fields are transactionally updating both main collection (already existing) and state collections (lazily created). Because of that, the first insert on a collection with FLE options is triggering the creation of the necessary state collections within a transaction.
Example of faulty flow:
- Create FLE collection
- Move it out the primary shard (moveChunk for sharded collection / moveCollection for unshareded collection)
- Insert data
- Write on the main collection will be routed to a shard different than the db primary
- Write on state collection(s) will target the primary shard and implicitly create the collection
- Transaction fails because trying to create collection in a distributed transaction
Possible solutions:
- Catch the error in fle write path and create FLE collections before retrying the transaction
- Always create FLE state collections as part of the main collection creation
Reproducible:
const st = new ShardingTest({shards: 2, mongos: 1, rs: {nodes: 1}}); const mongos = st.s0; const shard0 = st.shard0.shardName; const shard1 = st.shard1.shardName; const kDbName = 'test'; const kCollName = 'coll'; const kFullNs = kDbName + '.' + kCollName; const kms = { key: BinData( 0, "/tu9jUCBqZdwCelwE/EAm/4WqdxrSMi04B8e9uAV+m30rI1J2nhKZZtQjdvsSCwuI4erR6IEcEK+5eGUAODv43NDNIR9QheT2edWFewUfHKsl9cnzTc86meIzOmYl6dr") }; const csfleOpts = { kmsProviders: { local: kms, }, keyVaultNamespace: kDbName + ".keystore", schemaMap: {}, }; var shell = Mongo(mongos.host, csfleOpts); var kv = shell.getKeyVault(); const schema = { "fields": [ { "path": "foo", "keyId": kv.createKey("local", "ignored"), "bsonType": "string", "queries": {"queryType": "equality"} }, ] }; // Create FLE collection, shard it and move unique range out of the db primary shard const db = shell.getDB(kDbName); assert.commandWorked(db.createCollection(kCollName, {encryptedFields: schema})); assert.commandWorked(mongos.adminCommand({shardCollection: kFullNs, key: {_id: 1}})); const toShard = st.getPrimaryShard("test").name == shard0 ? shard1 : shard0; assert.commandWorked(mongos.adminCommand({moveChunk: kFullNs, find: {_id: 0}, to: toShard})); // This insert will fail db.getCollection(kCollName).insertOne({foo: "foovalue"}); st.stop();