[SERVER-34061] Stop preemptively loading sharded collections when loading a database into the CatalogCache Created: 22/Mar/18  Updated: 29/Oct/23  Resolved: 30/Jun/20

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.7.3
Fix Version/s: 4.7.0

Type: Task Priority: Major - P3
Reporter: Esha Maharishi (Inactive) Assignee: Tommaso Tocci
Resolution: Fixed Votes: 1
Labels: PM-1645-Milestone-2
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-48992 Implement database cache on top of Re... Closed
Duplicate
is duplicated by SERVER-44249 Shard can continue to accept reads fo... Closed
Problem/Incident
Related
related to SERVER-49196 Complete TODO listed in SERVER-34061 Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2020-06-29
Participants:
Case:
Linked BF Score: 10

 Description   

Ideally, refreshing a cached database entry would only involve reading from "config.databases".

But for historical reasons, refreshing a database also reloads the list of the database's sharded collections (throwing out their cached routing info) and marks them as needsRefresh.

Unfortunately, we can't stop loading the sharded collections on database refresh, because pending SERVER-32198, shards treat a missing cache entry to mean "unsharded."

If we did, if a test uses a stale mongos against a shard whose in-memory cache has been reset, the stale mongos will refresh for the database but not for the collection -> mongos will send shardVersion UNSHARDED -> the shard will also believe the collection is unsharded -> the test fails because some documents were not found, count is wrong, etc.

Specifically, these suites would fail:

  • read_only_sharded - because it restarts all shards + mongos, clearing their in-memory caches
  • concurrency_sharded_causal_consistency - because it does reads through a stale mongos against secondaries that have never loaded their cache into memory
  • concurrency_sharded_with_stepdowns - because it does reads through a stale mongos, and after a shard stepdown, the secondary that steps up has never loaded its cache into memory

Once SERVER-32198 is in, we should stop loading the sharded collections on database refresh.



 Comments   
Comment by Githook User [ 21/Jul/20 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-49196 Complete TODO listed in SERVER-34061
Branch: master
https://github.com/mongodb/mongo/commit/24f73a7d1a2923abcf1265cab4d56135cb61824a

Comment by Githook User [ 30/Jun/20 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-34061 Stop preemptively loading sharded collections when loading a database into the CatalogCache
Branch: master
https://github.com/mongodb/mongo/commit/ccb8ef3965b1ab95d8c8d1a1bda47b643a9e81cd

Comment by Esha Maharishi (Inactive) [ 22/Mar/18 ]

This is more important in 4.0, because database refresh can happen more frequently - after each movePrimary or dropDatabase.

Generated at Thu Feb 08 04:35:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.