[SERVER-77062] Perform copy-on-write of index catalog entries when cloning a collection Created: 12/May/23  Updated: 29/Oct/23  Resolved: 23/May/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 7.0.0-rc3

Type: Bug Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-77131 Remove and ban any write to collectio... Closed
Problem/Incident
causes SERVER-77842 Cache IndexCatalogEntry during the co... Closed
Related
related to SERVER-77082 Data race while setting index build i... Closed
is related to SERVER-77368 Remove const_cast<IndexAccessMethod*>... Closed
is related to SERVER-77379 Cleanup IndexCatalogImpl::_deleteInde... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0
Sprint: Execution Team 2023-05-29
Participants:
Linked BF Score: 144

 Description   

The consequence is that other collection instances that have this IndexCatalogEntry can observe changes from other operations that make changes to the IndexCatalogEntry.

TEST_F(CollectionCatalogTimestampTest, IndexCatalogEntryCopying) {
    const NamespaceString nss("test.abc");
    createCollection(opCtx.get(), nss, Timestamp::min());
    IndexCatalogEntry* entry;
    {
        // Start but do not finish an index build.
        IndexSpec spec;
        spec.version(1).name("x_1").addKeys(BSON("x" << 1));
        auto desc = std::make_unique<IndexDescriptor>(IndexNames::BTREE, spec.toBSON());
        AutoGetCollection autoColl(opCtx.get(), nss, MODE_X);
        WriteUnitOfWork wuow(opCtx.get());
        auto collWriter = autoColl.getWritableCollection(opCtx.get());
        ASSERT_OK(collWriter->prepareForIndexBuild(opCtx.get(), desc.get(), boost::none, false));
        entry = collWriter->getIndexCatalog()->createIndexEntry(
            opCtx.get(), collWriter, std::move(desc), CreateIndexEntryFlags::kNone);
        wuow.commit();
    }
    // In a different client, open the latest collection instance of 'nss'
    auto newClient = opCtx->getServiceContext()->makeClient("alternativeClient");
    auto newOpCtx = newClient->makeOperationContext();
    auto latestCatalog = CollectionCatalog::latest(newOpCtx.get());
    auto latestColl =
        latestCatalog->establishConsistentCollection(newOpCtx.get(), nss, boost::none);
    ASSERT_EQ(1, latestColl->getIndexCatalog()->numIndexesTotal());
    ASSERT_EQ(0, latestColl->getIndexCatalog()->numIndexesReady());
    ASSERT_EQ(1, latestColl->getIndexCatalog()->numIndexesInProgress());
    // The index is NOT ready.
    auto descLatestBefore = latestColl->getIndexCatalog()->findIndexByName(
        newOpCtx.get(),
        "x_1",
        IndexCatalog::InclusionPolicy::kReady | IndexCatalog::InclusionPolicy::kFrozen |
            IndexCatalog::InclusionPolicy::kUnfinished);
    auto entryLatestBefore = latestColl->getIndexCatalog()->getEntry(descLatestBefore);
    ASSERT(!entryLatestBefore->isReady());
    {
        // Now finish the index build on the original OperationContext.
        AutoGetCollection autoColl(opCtx.get(), nss, MODE_X);
        WriteUnitOfWork wuow(opCtx.get());
        auto collWriter = autoColl.getWritableCollection(opCtx.get());
        collWriter->getIndexCatalog()->indexBuildSuccess(opCtx.get(), collWriter, entry);
        wuow.commit();
    }
    // Container counts are still correct.
    ASSERT_EQ(1, latestColl->getIndexCatalog()->numIndexesTotal());
    ASSERT_EQ(0, latestColl->getIndexCatalog()->numIndexesReady());
    ASSERT_EQ(1, latestColl->getIndexCatalog()->numIndexesInProgress());
    // But the index is marked as ready now. When the collection clone happened above, it made a
    // copy of the containers, which is std::vector<std::shared_ptr<IndexCatalogEntry>> so we only
    // upped the ref count.
    auto descLatestAfter = latestColl->getIndexCatalog()->findIndexByName(
        newOpCtx.get(),
        "x_1",
        IndexCatalog::InclusionPolicy::kReady | IndexCatalog::InclusionPolicy::kFrozen |
            IndexCatalog::InclusionPolicy::kUnfinished);
    auto entryLatestAfter = latestColl->getIndexCatalog()->getEntry(descLatestAfter);
    ASSERT(!entryLatestAfter->isReady());
}



 Comments   
Comment by Githook User [ 30/May/23 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-77062 Perform copy-on-write of index catalog entries when cloning a collection

(cherry picked from commit b24cd4d58195e9c1a2055deacb25f86894ffbeff)
Branch: v7.0
https://github.com/mongodb/mongo/commit/e5e25d84b3843bed437061611854086c2e929bcb

Comment by Githook User [ 23/May/23 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-77062 Perform copy-on-write of index catalog entries when cloning a collection
Branch: master
https://github.com/mongodb/mongo/commit/b24cd4d58195e9c1a2055deacb25f86894ffbeff

Generated at Thu Feb 08 06:34:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.