[SERVER-55329] Move methods on ReshardingCollectionCloner into resharding_data_copy_util Created: 19/Mar/21  Updated: 29/Oct/23  Resolved: 25/Mar/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.0.0-rc0

Type: Task Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Max Hirschhorn
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-55330 Rewrite ReshardingOplogFetcher to use... Backlog
Problem/Incident
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2021-04-05
Participants:
Linked BF Score: 176
Story Points: 1

 Description   

Proposed interface:

resharding_data_copy_util.h

/**
 * Returns the largest _id value in the collection.
 */
Value findHighestInsertedId(OperationContext* opCtx, const CollectionPtr& collection);
 
/**
 * Returns a batch of documents totaling in size to at most batchSizeLimitBytes.
 */
std::vector<InsertStatement> fillBatchForInsert(Pipeline& pipeline, int batchSizeLimitBytes);
 
/**
 * Atomically inserts a batch of documents in a single storage transaction. Returns the number of
 * bytes inserted.
 *
 * Throws NamespaceNotFound if the collection doesn't already exist.
 */
int insertBatch(OperationContext* opCtx,
                const NamespaceString& nss,
                std::vector<InsertStatement>& batch);

resharding_collection_cloner.h

class ReshardingCollectionCloner {
public:
    ...
 
    /**
     * Schedules work to repeatedly fetch and insert batches of documents.
     *
     * Returns a future that becomes ready when either:
     *   (a) all documents have been fetched and inserted, or
     *   (b) the cancellation token was canceled due to a stepdown or abort.
     */
    SemiFuture<void> run(std::shared_ptr<executor::TaskExecutor> executor,
                         CancelationToken cancelToken);
 
    /**
     * Fetches and inserts a single batch of documents.
     *
     * Returns true if there are more documents to be fetched and inserted, and returns false
     * otherwise.
     */
    bool doOneBatch(OperationContext* opCtx, Pipeline& pipeline);
 
private:
    std::unique_ptr<Pipeline, PipelineDeleter> _restartPipeline(OperationContext* opCtx);
 
    ...
};



 Comments   
Comment by Githook User [ 25/Mar/21 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-55329 Destroy executor after ReshardingCollectionCloner::run().
Branch: master
https://github.com/mongodb/mongo/commit/d766d76653846b5a2c21e0bb1ac590d2d1fb299b

Comment by Githook User [ 23/Mar/21 ]

Author:

{'name': 'Max Hirschhorn', 'email': 'max.hirschhorn@mongodb.com', 'username': 'visemet'}

Message: SERVER-55329 Move batch insert logic into resharding_data_copy_util.
Branch: master
https://github.com/mongodb/mongo/commit/dd57a991923a3f93220407c74a43b71574b771be

Generated at Thu Feb 08 05:36:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.