[SERVER-30148] Move force primary refresh functionality out of the ShardServerCatalogCacheLoader into the ShardingState's refresh logic Created: 14/Jul/17  Updated: 06/Dec/22  Resolved: 17/Dec/21

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: [DO NOT USE] Backlog - Sharding EMEA
Resolution: Won't Do Votes: 0
Labels: ShardingTechDebt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-32016 Create an interruptible class/data st... Closed
Gantt Dependency
has to be done before SERVER-30550 Test that safe secondary reads is ina... Closed
has to be done before SERVER-30640 Make secondary mode unit tests for Sh... Closed
has to be done after SERVER-30147 Add collection task queue flush funct... Closed
Related
related to SERVER-62159 Complete TODO listed in SERVER-30148 Closed
is related to SERVER-25359 Create collection-specific ResourceMu... Closed
is related to SERVER-31595 Generate shardMaps outside MODE_X col... Closed
Assigned Teams:
Sharding EMEA
Sprint: Sharding 2017-12-04
Participants:

 Description   

We want to simplify the ShardServerCatalogCacheLoader by making a simple local loader and reader of chunk metadata. So we want to move the remote call to ensure the primary has refreshed to the latest metadata out of the ShardServerCatalogCacheLoader and instead place this functionality at a higher level in the refresh logic in ShardingState::onStaleShardVersion. A rough sketch:

onStaleShardVersion(wantedVersion) {
    while () {
        invalidateCatalogCacheMetadata
        refreshAndLoadLocalChunkMetadata
        if (wantedVersion > locallyRefreshedVersion) {
            optime = forcePrimaryRefresh
            waitForOptime(optime)
        }
    }
}



 Comments   
Comment by Dianna Hohensee (Inactive) [ 15/Nov/17 ]

We're considering creating a new interruptible class/data structure, somewhat similar to the LockManager, except it just handles serializing operations per namespace.

Linking SERVER-25359 because such a thing would make its task simple.

Comment by Dianna Hohensee (Inactive) [ 14/Nov/17 ]

Given the fact that ShardingState would seem to need to know primary versus secondary status in order to do the correct operations, and the difficulty of preventing all waiting callers from all sending the forcePrimaryRefresh command (redundantly), I'm wondering about alternatives. We could combine making oplog invalidations kick off the refresh process immediately, and best effort informing the recipient shard of its new chunk, and get pretty solid availability that way.

Comment by Dianna Hohensee (Inactive) [ 14/Jul/17 ]

Must be done after SERVER-30147, which changes the forceRoutingTableRefresh command to return an optime.

Generated at Thu Feb 08 04:22:49 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.