[SERVER-72530] $_internalOwningShard aggregation expression should not cause a refresh of the sharding filtering metadata Created: 05/Jan/23  Updated: 29/Oct/23  Resolved: 04/Jul/23

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Pol Pinol
Resolution: Fixed Votes: 0
Labels: shardingemea-qw
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-78379 Remove out-dated $_internalOwningShar... Open
Related
is related to SERVER-68686 Add new $_internalOwningShard agg exp... Closed
Assigned Teams:
Sharding EMEA
Backwards Compatibility: Fully Compatible
Sprint: Sharding EMEA 2023-01-23, Sharding EMEA 2023-06-12, Sharding EMEA 2023-06-26, Sharding EMEA 2023-07-10
Participants:

 Description   

The $_internalOwningShard aggregation expression from SERVER-68686 is designed to be used to make routing decisions of the form [given an input document from collection Source, to what shard for collection Destination should the document be inserted into?]. Its usage assumes sharding_ddl_util::stopMigrations() was already run to ensure the placement information doesn't continue to change while the aggregation is being run. And, for the intended usage of the $_internalOwningShard aggregation expression in resharding a collection or building a global index, the split points + ownership for the new collection / global index container won't change until those operations entirely complete.

The $_internalOwningShard aggregation expression therefore accepts a shard version to ensure the CatalogCache has been refreshed up until which provides the necessary causal consistency of the routing table information. In the case where the CatalogCache is stale, ExpressionInternalOwningShard throws a StaleConfig exception.

// Invalidate catalog cache if the chunk manager version is stale.
if (chunkManager.getVersion().isOlderThan(shardVersion.placementVersion())) {
    boost::optional<CollectionIndexes> collIndexes;
    ShardVersion currentShardVersion(chunkManager.getVersion(), collIndexes);
    uasserted(StaleConfigInfo(ns,
                              currentShardVersion,
                              boost::none /* wanted */,
                              ShardingState::get(opCtx)->shardId()),
              str::stream()
                  << "Sharding information of collection " << ns
                  << " is currently stale and needs to be recovered from the config server");
}

While it may be the case that forcing a refresh of the sharding filter metadata induces a refresh of the routing information for the collection, this feels like an implementation detail that shouldn't be relied upon. It would be more direct and likely to continue being supported if ExpressionInternalOwningShard caused the routing information for the collection to be refreshed by doing

  1. Call catalogCache->invalidateShardOrEntireCollectionEntryForShardedCollection(ns, shardVersion, ShardId()).
  2. Throw a ShardCannotRefreshDueToLocksHeld exception to cause the shard to refresh (only) its CatalogCache.


 Comments   
Comment by Githook User [ 04/Jul/23 ]

Author:

{'name': 'Pol Pinol Castuera', 'email': 'pol.pinol@mongodb.com', 'username': 'PolPinol'}

Message: SERVER-72530 Make $_internalOwningShard aggregation not cause a refresh of the sharding filtering metadata
Branch: master
https://github.com/mongodb/mongo/commit/9887f187ef46151d33dd895be8503f2ad0fe8d57

Generated at Thu Feb 08 06:22:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.