[SERVER-27052] Add asynchronous operation support to DataReplicator Created: 15/Nov/16  Updated: 25/Apr/18  Resolved: 06/Dec/16

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.4.0-rc3
Fix Version/s: 3.4.2, 3.5.1

Type: Improvement Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File 0001-SERVER-27052-added-asynchronous-operation-support-to.patch    
Issue Links:
Backports
Depends
depends on SERVER-27079 DataReplicator should wait for last o... Closed
Problem/Incident
causes SERVER-30913 remove unused getLatestOplogEntry met... Closed
Related
related to SERVER-25966 Add initial sync unittests for metada... Closed
related to SERVER-26704 Refactor DataReplicator::getLatestOpl... Closed
related to SERVER-27047 DataReplicator should return Shutdown... Closed
related to SERVER-27134 Do not reset numGetMoreOplogEntries i... Closed
related to SERVER-27135 Check if DataReplicator finishEvent h... Closed
related to SERVER-31995 Logged initial sync statistics may ex... Closed
is related to SERVER-25662 Data replicator should stop initial s... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.4
Sprint: Repl 2016-11-21, Repl 2016-12-12
Participants:
Linked BF Score: 0

 Description   

DataReplicator currently requires a dedicated thread to run doInitialSync, which has ramifications for testing and makes it inconvenient to integrate with other components such as the ReplicationCoordinator.



 Comments   
Comment by Githook User [ 22/Dec/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 added asynchronous operation support to DataReplicator

(cherry picked from commit 2068c42aa2179902d4a96941fcfc7cd577a4c2a9)
(cherry picked from commit 5313489266e75c5d40d1b7aae382d3e3fc8997a0)
(cherry picked from commit 06da357873b3500f39832dee914c06b1968d05ca)
(cherry picked from commit 19a7178e9b457d75b0431f384b85d69d7309a563)
(cherry picked from commit a73e65ba511d047431b85138b010818800b7de04)
(cherry picked from commit 2113ca6c7a18f109aad1e11200e8ab034bcd78fa)
(cherry picked from commit bed3e78f3c511752d894f9449b93f12992a9eb3c)
(cherry picked from commit a8ce641a86cff734d3e3756dbfd1beb892f639ba)
(cherry picked from commit b1a5aefc4467c5cff6e7bcf3f4aef3cb15f3ef05)
(cherry picked from commit e30e39ce1a4a55c46db13ad85f6c1000297ea6ff)
(cherry picked from commit 39d11435c22a8c6ce2c489221102605a9185e815)
(cherry picked from commit 1f9513ef67551db6ea93b8b9e2f40604167f952b)
Branch: v3.4
https://github.com/mongodb/mongo/commit/a89e9c71a7ff29a652afbc837cc3e26d80fc17a0

Comment by Benety Goh [ 07/Dec/16 ]

(3.4 backport) attached "git am" patch file is created using base commit 66edc250f46519ea7c6b65f628d201ffbfd0dfbf with SERVER-27079 cherry-picked into 3.4

Comment by Githook User [ 06/Dec/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 fixed FetcherTest::ScheduleButShutdown test case - test should not assume when the task executor's shutdown() function will deliver the cancellation event to the fetcher's callback
Branch: master
https://github.com/mongodb/mongo/commit/1f9513ef67551db6ea93b8b9e2f40604167f952b

Comment by Githook User [ 06/Dec/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 fixed lint
Branch: master
https://github.com/mongodb/mongo/commit/39d11435c22a8c6ce2c489221102605a9185e815

Comment by Githook User [ 06/Dec/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 added asynchronous operation support to DataReplicator
Branch: master
https://github.com/mongodb/mongo/commit/e30e39ce1a4a55c46db13ad85f6c1000297ea6ff

Comment by Githook User [ 30/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 MultiApplier clears MultiApplier::_onCompletion on completion to release any resources that might be held by function object
Branch: master
https://github.com/mongodb/mongo/commit/b1a5aefc4467c5cff6e7bcf3f4aef3cb15f3ef05

Comment by Githook User [ 30/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 OplogFetcher clears OplogFetcher::_onShutdownCallbackFn on completion to release any resources that might be held by function object
Branch: master
https://github.com/mongodb/mongo/commit/a8ce641a86cff734d3e3756dbfd1beb892f639ba

Comment by Githook User [ 30/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 make Fetcher single-use only - cannot be restarted after failing at startup or completed processing remote command responses.

Fetcher clears Fetcher::_work on completion to release any resources that might be held by function object
Branch: master
https://github.com/mongodb/mongo/commit/bed3e78f3c511752d894f9449b93f12992a9eb3c

Comment by Githook User [ 30/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 make CollectionCloner single-use. CollectionCloner should also free up resources held by _onCompletion callback when it finishes
Branch: master
https://github.com/mongodb/mongo/commit/2113ca6c7a18f109aad1e11200e8ab034bcd78fa

Comment by Githook User [ 29/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 fixed DataReplicatorPassesThroughRollbackCheckerScheduleError test to reject only the replSetGetRBID schedule request

This commit addresses an issue introduced in 06da357873b3500f39832dee914c06b1968d05ca.
Branch: master
https://github.com/mongodb/mongo/commit/a73e65ba511d047431b85138b010818800b7de04

Comment by Githook User [ 29/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 clarified DatabasesCloner's startup and shutdown contract.

DatabasesCloner::startup() does not invoke completion callback on failing to schedule first remote command
DatabasesCloner::shutdown() leaves _active state unchanged
DatabasesCloner clears DatabasesCloner::_finishFn on completion to release any resources that might be held by function object
make DatabasesCloner single-use only - cannot be restarted once completed
Branch: master
https://github.com/mongodb/mongo/commit/19a7178e9b457d75b0431f384b85d69d7309a563

Comment by Githook User [ 29/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 changed RollbackChecker::reset() and checkForRollback() to pass through result from TaskExecutor::scheduleRemoteCommand
Branch: master
https://github.com/mongodb/mongo/commit/06da357873b3500f39832dee914c06b1968d05ca

Comment by Githook User [ 29/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 moved SyncSourceSelectorMock to its own library
Branch: master
https://github.com/mongodb/mongo/commit/5313489266e75c5d40d1b7aae382d3e3fc8997a0

Comment by Githook User [ 28/Nov/16 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-27052 ThreadPoolTaskExecutor clears 'cbStateArg->callback' after running callback function to release any resources that might be held by function object
Branch: master
https://github.com/mongodb/mongo/commit/2068c42aa2179902d4a96941fcfc7cd577a4c2a9

Generated at Thu Feb 08 04:14:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.