[SERVER-34900] initial sync uses different batch limits from steady state replication Created: 08/May/18  Updated: 29/Oct/23  Resolved: 10/May/18

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.4.14, 3.6.4, 3.7.9
Fix Version/s: 3.6.6, 4.0.0-rc0

Type: Bug Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-34942 Stuck with cache full during oplog re... Closed
is related to SERVER-36496 Cache pressure issues during oplog re... Closed
is related to SERVER-32335 Update initial sync to use new Applie... Closed
is related to SERVER-35405 Change default setting for replBatchL... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.6, v3.4
Sprint: Repl 2018-05-21
Participants:

 Description   

Initial sync limits batches during oplog application to 5000 ops/dur::UncommittedBytesLimit (512 MB) while steady state replication has been limiting batches to 50,000 ops/100MB since commit b53a93270c8ea8ccc9f43a0cf0592353b0edd012 (SERVER-24242).

Getting the batch limits from SyncTail will also allow us to tweak the maximum number of operations in each batch during initial sync using the server parameter replBatchLimitOperations.



 Comments   
Comment by Githook User [ 14/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 initial sync uses same batch limits as steady state replication

(cherry picked from commit b3b4616f44ec2b0c820ca53e469c333d954c8ce6)
Branch: v3.6
https://github.com/mongodb/mongo/commit/ba66bc7f1807f0a45d2b841f6af689e0cd2ada26

Comment by Kelsey Schubert [ 05/Jun/18 ]

The functional change has not yet been backported to 3.6

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 add ReplicationCoordinatorExternalState::getInitialSyncBatchLimits()

(cherry picked from commit 3bee8adabe9841da544a7e46788cbf5143552bb6)
Branch: v3.6
https://github.com/mongodb/mongo/commit/e3771d986d7987e25be0c8e67adfdf8315c1deb6

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 initial sync uses SyncTail::BatchLimits to hold oplog application batching settings

(cherry picked from commit 4b8abbe03c7acb16a48a750af3253e7aed330e5d)
Branch: v3.6
https://github.com/mongodb/mongo/commit/e0abafe82c4757b1080632b3edd7b622694c284b

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 BatchLimits construction should not reference SyncTail runtime server parameters

(cherry picked from commit 8e193517d7c65070ec4d302aff83e18be56e96e8)
Branch: v3.6
https://github.com/mongodb/mongo/commit/198ff88cb2259e4ce1aa98b827d6e3d86379a9f7

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 make batch limit byte and operation count defaults public

(cherry picked from commit 39a0d0bea4335b2eb91eaefb670de772425628fd)
Branch: v3.6
https://github.com/mongodb/mongo/commit/4f8037dd5a7d686baf055dbd8bde0fa93594f8ed

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-34900 remove unused constant SyncTail::replBatchLimitSeconds

Last reference to this constant was removed in commit b06901cd83b2a985aa50f9a699f3d63dcd28476d.

(cherry picked from commit 0a05a0a1c25a6f226c5d14cf72e8394b9d1e8951)
Branch: v3.6
https://github.com/mongodb/mongo/commit/0bd64063334638bd074192b33293060a5a30751f

Comment by Githook User [ 10/May/18 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-34900 initial sync uses same batch limits as steady state replication
Branch: master
https://github.com/mongodb/mongo/commit/b3b4616f44ec2b0c820ca53e469c333d954c8ce6

Comment by Githook User [ 10/May/18 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-34900 add ReplicationCoordinatorExternalState::getInitialSyncBatchLimits()
Branch: master
https://github.com/mongodb/mongo/commit/3bee8adabe9841da544a7e46788cbf5143552bb6

Comment by Githook User [ 09/May/18 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-34900 initial sync uses SyncTail::BatchLimits to hold oplog application batching settings
Branch: master
https://github.com/mongodb/mongo/commit/4b8abbe03c7acb16a48a750af3253e7aed330e5d

Comment by Githook User [ 09/May/18 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-34900 BatchLimits construction should not reference SyncTail runtime server parameters
Branch: master
https://github.com/mongodb/mongo/commit/8e193517d7c65070ec4d302aff83e18be56e96e8

Comment by Githook User [ 09/May/18 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-34900 make batch limit byte and operation count defaults public
Branch: master
https://github.com/mongodb/mongo/commit/39a0d0bea4335b2eb91eaefb670de772425628fd

Comment by Githook User [ 09/May/18 ]

Author:

{'email': 'benety@mongodb.com', 'name': 'Benety Goh', 'username': 'benety'}

Message: SERVER-34900 remove unused constant SyncTail::replBatchLimitSeconds

Last reference to this constant was removed in commit b06901cd83b2a985aa50f9a699f3d63dcd28476d.
Branch: master
https://github.com/mongodb/mongo/commit/0a05a0a1c25a6f226c5d14cf72e8394b9d1e8951

Generated at Thu Feb 08 04:38:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.