[SERVER-40346] Write shardCollection initial chunks with BatchWriter Created: 26/Mar/19  Updated: 29/Oct/23  Resolved: 16/Apr/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 4.0.7, 4.1.9
Fix Version/s: 4.1.10, 4.0.11

Type: Improvement Priority: Major - P3
Reporter: Blake Oler Assignee: Blake Oler
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Sharding 2019-04-08, Sharding 2019-04-22
Participants:

 Description   

Problem Statement

Currently, when we shard a collection, we write each chunk document to the config server sequentially, using a {w:majority} write.

This means that for 10,000 chunks we will make 10,000 calls from a shard to the config server. We can prevent this by batch writing these documents instead.

Proposed Solution

  1. Make an AlternativeSessionRegion class like the AlternativeClientRegion to control the lifetime of a generated session RAII-style. This class will take in either a session or generate its own. It will stash the current session on the thread and replace with the new session. On destruction, it will replace the session with the original, and destroy the newly created session (but only if the class created the session itself). For now, it may be easier to restrict the class to only self-generate sessions, so I could be persuaded to do that as well.
  2. Make a new function in the ShardingCatalogClient: insertConfigDocumentsAsRetryableWrite(). This function will take in a vector of BSONObj documents, a namespace, and a transaction number. It will run a batch command to insert them with the idempotent retry policy.
  3. Change InitialSplitPolicy::writeFirstChunksToConfig() to run under the AlternativeSessionRegion and insert chunk documents in bulk fashion.


 Comments   
Comment by Githook User [ 04/Jun/19 ]

Author:

{'name': 'Blake Oler', 'email': 'blake.oler@mongodb.com', 'username': 'BlakeIsBlake'}

Message: SERVER-40346 Use AlternativeSessionRegion to insert config documents as retryable write

(cherry picked from commit 29ef1a415c74c883746325f13a8eaaa1831f8102)
Branch: v4.0
https://github.com/mongodb/mongo/commit/0177acdb774e1808b01f1b2139420fabb304694d

Comment by Githook User [ 16/Apr/19 ]

Author:

{'name': 'Blake Oler', 'username': 'BlakeIsBlake', 'email': 'blake.oler@mongodb.com'}

Message: SERVER-40346 Use AlternativeSessionRegion to insert config documents as retryable write
Branch: master
https://github.com/mongodb/mongo/commit/29ef1a415c74c883746325f13a8eaaa1831f8102

Comment by Kaloian Manassiev [ 03/Apr/19 ]

This approach LGTM.

For the AlternativeSessionRegion class, in order to avoid people finding new "clever" usages of it, can you please keep it contained to the file where insertConfigDocumentsAsRetryableWrite will be implement?

Comment by Blake Oler [ 01/Apr/19 ]

kaloian.manassiev can I get an LGTM on the approach?

Generated at Thu Feb 08 04:54:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.