[SERVER-47438] Cloner::copyIndexes() should generate createIndexes oplog entry (single phase index build) for building indexes on empty collection. Created: 09/Apr/20  Updated: 29/Oct/23  Resolved: 16/Apr/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.4.0-rc2, 4.7.0

Type: Task Priority: Major - P3
Reporter: Suganthi Mani Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
is related to SERVER-26772 Remove old initial sync code Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4
Sprint: Execution Team 2020-04-20
Participants:

 Description   

This Cloner::copyIndexes() is called either during rollback via refetch or by sharding specific commands "_shardsvrCloneCatalogData" & "_cloneCatalogData".

If this Cloner::copyIndexes() is called on primary for creating indexes on a non-empty collection, this can cause secondaries to block replication. But, it seems, if the node is primary, then we build indexes only on empty collections. To make this reasoning more obvious to the readers, we should make sure that if writes are replicated
1) Node should be in primary
2) Indexes that being built should be only on empty collection (can use IndexBuildsCoordinator::createIndexesOnEmptyCollection()) and they generate createIndexes oplog entry, making it more like a single phase.

If writes are not replicated, the collection may not be empty. This can happen only during rollback via refetch and we don't generate any oplog entries.



 Comments   
Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::_copyIndexes() will not recreate dropped collection

(cherry picked from commit 44af4b2915cef5afcfe63104fb902e4519ffa889)
Branch: v4.4
https://github.com/mongodb/mongo/commit/68e3f4ee266273a9dad7ccfd5cbed5e4d9bcb324

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::_copyIndexes() assumes collection is empty

(cherry picked from commit 03793e2a6a59566858f993598a3eb924def58a1a)
Branch: v4.4
https://github.com/mongodb/mongo/commit/62fa038f57216e5773e23266fa366236f94a5b4b

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::copyDb() creates indexes before copying documents

(cherry picked from commit 555a44b7bb6f0fea7e215283fa805f6ae6a6e850)
Branch: v4.4
https://github.com/mongodb/mongo/commit/deef7073368a05721823e86b5ab396095687e60e

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::copyDb() always copies the same database from a remote host

(cherry picked from commit fc99a1a80f0d4cc063d47db3cacbba462bf69154)
Branch: v4.4
https://github.com/mongodb/mongo/commit/eeaaaa171663cc96c3eed18f43f4e3e9e233fa90

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove CloneOptions. Cloner::copyDb() accepts db name and sharded collection names

(cherry picked from commit a01a444b062f7911e47d9ce3b6b4206622e96dc9)
Branch: v4.4
https://github.com/mongodb/mongo/commit/91048cc39bdca1c92cb2f4626980924e5b949b2a

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove CloneOptions::slaveOk

(cherry picked from commit bf5c17f39148ee5c0c113d4236b76b7dd919c877)
Branch: v4.4
https://github.com/mongodb/mongo/commit/c962474496c70b853258a0b9527838efa59499be

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove Cloner::copyCollection() and Cloner::setConnection()

These functions were previously required by rollback via refetch.

(cherry picked from commit 6388b971ca8df0d0e4b2fe4ccfc4ecc9a10decca)
Branch: v4.4
https://github.com/mongodb/mongo/commit/8fad63a609a2beb863f276fa4281074719053df8

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove unused CloneOptions fields

Most recent references were removed in commits f53a88a57da5788b355cb3037061372a706ccf0d and 64e649a622b5ac4c1bfad0933132dc7d994b9458.

(cherry picked from commit eafda6eb37ce6681285a5fbc650b0c1f9b8a4924)
Branch: v4.4
https://github.com/mongodb/mongo/commit/e10ed5174e56ecdb93009d7983896daa76211efd

Comment by Githook User [ 17/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove unused function RollbackSource::copyCollectionFromRemote()

Last reference was removed in commit 65e6cd5142082eab94a561036a53abdf60139155.

(cherry picked from commit d7f09737cb3354b45368c373169156140b933773)
Branch: v4.4
https://github.com/mongodb/mongo/commit/e1fadc7f894645da2e42bd5145e9731e528e99c0

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::_copyIndexes() will not recreate dropped collection
Branch: master
https://github.com/mongodb/mongo/commit/44af4b2915cef5afcfe63104fb902e4519ffa889

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::_copyIndexes() assumes collection is empty
Branch: master
https://github.com/mongodb/mongo/commit/03793e2a6a59566858f993598a3eb924def58a1a

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::copyDb() creates indexes before copying documents
Branch: master
https://github.com/mongodb/mongo/commit/555a44b7bb6f0fea7e215283fa805f6ae6a6e850

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 Cloner::copyDb() always copies the same database from a remote host
Branch: master
https://github.com/mongodb/mongo/commit/fc99a1a80f0d4cc063d47db3cacbba462bf69154

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove CloneOptions. Cloner::copyDb() accepts db name and sharded collection names
Branch: master
https://github.com/mongodb/mongo/commit/a01a444b062f7911e47d9ce3b6b4206622e96dc9

Comment by Githook User [ 16/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove CloneOptions::slaveOk
Branch: master
https://github.com/mongodb/mongo/commit/bf5c17f39148ee5c0c113d4236b76b7dd919c877

Comment by Githook User [ 15/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove Cloner::copyCollection() and Cloner::setConnection()

These functions were previously required by rollback via refetch.
Branch: master
https://github.com/mongodb/mongo/commit/6388b971ca8df0d0e4b2fe4ccfc4ecc9a10decca

Comment by Githook User [ 15/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove unused CloneOptions fields

Most recent references were removed in commits f53a88a57da5788b355cb3037061372a706ccf0d and 64e649a622b5ac4c1bfad0933132dc7d994b9458.
Branch: master
https://github.com/mongodb/mongo/commit/eafda6eb37ce6681285a5fbc650b0c1f9b8a4924

Comment by Githook User [ 15/Apr/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-47438 remove unused function RollbackSource::copyCollectionFromRemote()

Last reference was removed in commit 65e6cd5142082eab94a561036a53abdf60139155.
Branch: master
https://github.com/mongodb/mongo/commit/d7f09737cb3354b45368c373169156140b933773

Comment by Benety Goh [ 09/Apr/20 ]

A lot of the Cloner logic was made obsolete with SERVER-26772.

Generated at Thu Feb 08 05:14:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.