[SERVER-6497] provide an option to deny global queries (queries running on all shards) Created: 17/Jul/12  Updated: 12/Dec/23

Status: Backlog
Project: Core Server
Component/s: Admin, Sharding
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Alon Horev Assignee: Backlog - Cluster Scalability
Resolution: Unresolved Votes: 3
Labels: sharding-nyc-subteam2
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Cluster Scalability
Participants:
Story Points: 4

 Description   

global queries can be sent innocently to a large cluster and hurt it's overall performance. In many systems global queries are intentionally avoided in order to make the system truly scalable (see a more elaborate explanation about the scale issue in the next paragraph).
This issue attempts to request a collection level configuration that won't allow global queries, in other words, require the query to contain a shard key.

Why global queries don't scale?:
Usually there is a correlation between the size of the data and the number of queries per second invoked. Take for example a social network implementation: you start off with one shard and 1000 users. when you find yourself with 50 shards and 50000 you make 50 times more queries. if queries are global, the more users you have, the more shards you have, more queries are invoked per shard. I've heard MongoDB devs say before: each shard is doing 1/50 of the work. well that might be true for some map reduce queries, but not for many other simple queries.
Another killer example is the system i'm working on: A 50 shard cluster scattered across the U.S, do I really want to suffer cross data center latencies for each query? I think not.


Generated at Thu Feb 08 03:11:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.