When a chunk is migrated between shards, the donor shard retains the migrated documents as orphans until the range deletion task cleans them up. While these orphans are present, a write that targets the donor shard and re-uses the _id of an orphan fails with a duplicate key error (code 11000) on the _id index, even though the operation is legitimate from the user's point of view (the _id is not visible in the collection through mongos).

The root cause of both scenarios below is the same: the presence of orphaned documents on a shard causes writes (inserts, or shard key updates that internally perform an insert) to fail with a duplicate key error on _id.

Scenario 1: Shard key update

When updating a document's shard key in a way that requires moving the document to a different shard, the operation deletes the document from the current shard and inserts it on the shard owning the target key range, inside a transaction. The issue arises when the target shard of that insert is the same shard that previously donated the chunk containing the document and still retains the orphaned documents. When the shard key update then tries to insert the document with its new shard key value on that shard, it hits a duplicate key error on _id due to the orphan. The operation fails with error code 11000 and the message: "Failed to update document's shard key field. There is either an orphan for this document or _id for this collection is not globally unique. :: caused by :: During insert stage of updating a shard key :: caused by :: E11000 duplicate key error collection: test.foo index: id dup key: { _id: 1.0 }".

Scenario 2: Delete followed by insert re-using the same _id

The same underlying problem is reachable with plain inserts, without any shard key update:

• A collection is sharded on {skey: 1}. Shard0 owns ranges [MinKey, 0) and [0, MaxKey), and holds a document {_id: 10, skey: 20} in [0, MaxKey).
• The chunk [0, MaxKey) is migrated from shard0 to shard1. Shard0 retains {_id: 10, skey: 20} as an orphan (range deletion not yet run).
• The user deletes {_id: 10} (routed to shard1, the owner of the live document). No document with _id: 10 is now visible in the collection.
• The user inserts a new document {_id: 10, skey: -5}. Its shard key belongs to [MinKey, 0), still owned by shard0. The insert fails with a duplicate key error on _id because shard0 still holds the orphan {_id: 10, skey: 20}.

This is unexpected: from the user's perspective there is no _id duplicate, and the _id uniqueness contract is honored across the visible documents.