[SERVER-55167] Investigate why tenant migrations can get DuplicateKey error when run with background_unique_indexes.js Created: 11/Mar/21  Updated: 27/Oct/23  Resolved: 15/Mar/21

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Cheahuychou Mao Assignee: Jason Chan
Resolution: Gone away Votes: 0
Labels: pm-1791_other_required
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-55157 Make tenant migration hook throw an e... Closed
Operating System: ALL
Sprint: Repl 2021-03-22
Participants:

 Description   

Tenant migrations in this test consistently abort due to DuplicateKey error as shown below.

{'state': 'aborted', 'abortReason': {'code': 11000, 'codeName': 'DuplicateKey', 'errmsg': 'Tenant migration recipient command failed :: caused by :: E11000 duplicate key error collection: tenantMigrationTenantId_background_unique_indexes.test index: x_1 dup key: { x: 791.0 }', 'keyPattern': {'x': 1.0}, 'keyValue': {'x': 791.0}}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1615490263, 514), 'signature': {'hash': b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 'keyId': 0}}, 'operationTime': Timestamp(1615490263, 514)}



 Comments   
Comment by Jason Chan [ 11/Mar/21 ]

The following scenario is happening:

1. donor collection with unique index on x: {id: 1, x: 1}, {id: 2, x: 2}, {id: 3, x: 3}
2. recipient starts cloning.
3. donor applies: $inc in order of id: 3, 2, 1
4. recipient cloning finishes. Starts oplog application
5. recipient starts applying the $inc but not in order. eg. apply $inc on id: 1 before id: 2, then we will get duplicate key error
 

This is a problem for unique indexes on secondaries as addressed on SERVER-38675.

Now that we do oplog application on primaries too, we should also relax the index constraints during tenant oplog application. This should be handled in SERVER-48593.

Generated at Thu Feb 08 05:35:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.