[DOCS-910] Discuss caveat of batchInsert through mongos Created: 19/Dec/12  Updated: 30/Oct/23  Resolved: 19/Dec/12

Status: Closed
Project: Documentation
Component/s: manual
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Task Priority: Minor - P4
Reporter: Jeremy Mikola Assignee: Sam Kleinman (Inactive)
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates DOCS-766 Sharded bulk insert in 2.0/2.2 mongos... Closed
Participants:
Days since reply: 11 years, 9 weeks ago

 Description   

User experienced lousy batchInsert performance through mongos. By default, batchInsert is supposed to stop if a document in the batch would trigger an error. In the case of mongos, this means that a single batch gets split up into sequences per shard. For instance, the following would translate into three sub-batches:

  • Document A destined for shard1
  • Document B destined for shard1
  • Document C destined for shard2
  • Document D destined for shard1

mongos can peak ahead in the batch and send multiple documents in appearing in sequence, but it can't send D to shard1 until C has been successfully inserted on shard2 – and that, of course, requires shard1 to acknowledge A and B. An absolute worst case would be a batch requiring exactly N distinct insert operations across a bunch of shards, where N is the total size of the batch (this assumes the largest sequence size was 1).

Using the continueOnError option can probably mitigate this issue, since that allows us to insert successive documents without checking for errors.


Generated at Thu Feb 08 07:39:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.