[SERVER-64980] Collectionless $listCatalog should not return catalog entries from non-chunk-owning shards Created: 28/Mar/22 Updated: 05/Feb/24 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Gregory Noma | Assignee: | Backlog - Catalog and Routing |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | car-qw | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Assigned Teams: |
Catalog and Routing
|
||||||||||||
| Sprint: | CAR Team 2023-12-11, CAR Team 2023-12-25, CAR Team 2024-01-08, CAR Team 2024-01-22 | ||||||||||||
| Participants: | |||||||||||||
| Story Points: | 1 | ||||||||||||
| Description |
|
Currently, the collectionless version of $listCatalog blindly returns all catalog entries from every shard. However, if a shard does not contain any chunks for a given collection, it would be better to exclude the catalog entry from that shard for that collection. |
| Comments |
| Comment by Gregory Noma [ 11/Dec/23 ] |
|
Even if it's expensive, my take is that it would still be worth the cost. I think it's very confusing that the two versions of $listCatalog have different semantics. And certain users of the feature currently have to jump through some hoops of using both versions of $listCatalog in a dance because of this behavior. In fact the collection-level $listCatalog was only ever added because of this issue. |
| Comment by Josef Ahmad [ 05/Dec/23 ] |
|
gregory.noma@mongodb.com apologies for the delay in getting back to you. My initial comment was during triage, where I needed more context. I have now observed that the collection-specific listCatalog invocation (e.g. db.getSiblingDB("test").c.aggregate([\{$listCatalog: {}}])) filters out shards that don't own any range, unlike the collectionless invocation (db.getSiblingDB("admin").aggregate([\{$listCatalog: {}}])) which doesn't seem to do ownership checks on collections. We will investigate the cost of adding that logic, although I suspect it could be quite expensive to do it at scale. |
| Comment by Gregory Noma [ 23/Oct/23 ] |
|
josef.ahmad@mongodb.com could you clarify what you mean by "cluster-wide catalog awareness"? |
| Comment by Josef Ahmad [ 23/Oct/23 ] |
|
As part of this ticket we should revisit whether we want to bring cluster-wide catalog awareness to $listCatalog, which has historically been local-only. If not, let's close it out. |