[SERVER-68341] Implement enable/disable command for mongoQ in serverless Created: 26/Jul/22 Updated: 12/Dec/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Rishab Joshi (Inactive) | Assignee: | Backlog - Query Execution |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | change-streams-w/-mongoq, phase3, pm3278-m2, replace-atlas-proxy-w-mongoq | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Query Execution
|
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
PM-2334 implemented enable/disable change streams command for the replica sets. The corresponding ticket was: This work for the sharded cluster was punted because there are subtle differences in how mongoS and mongoQ behave. For example, in mongoQ, the above command should dispatch the request to only a subset of shards where the corresponding tenant resides as opposed to mongoS where the request gets sent to all the shards. Since, there can be up to 10k shards (nodes) in the serverless, fanning out requests blindly to all the nodes is not a scalable solution. This ticket is about extending the work done in the
Since we will be building this enable/disable command using the POS service, we don't have atomicity when creating change collections on shards, ie. a change collection at shard1 might be made a timestamp t1 and on shard2 at timestamp t2. Whilst in general, this is not a problem for the change stream, it could be a problem for the following rare occurring scenario: Consider this:
Above is not a problem for the oplog because the oplog gets created at a very early stage of the node-bring up, as such the change stream resume token is always greater than the first entry of all the oplog in all the nodes. To mitigate above issue, we could potentially enable a change stream in the mongoQ in 2 stages -
With this, we will ensure that the resume token that the client will operate upon will be greater than the initial timestamp of all the change collections. While this is just one solution, we will explore more options when we work on this ticket.
This work will be under the serverless initiative - Change streams with mongoq |
| Comments |
| Comment by Kyle Suarez [ 14/Apr/23 ] |
|
Had a quick discussion with bernard.gorman@mongodb.com and while it's not quite the same motivation as the rest of the tickets in the mongos process interface project, it's fine to just tack it on to that project. Removing this from director triage. Notes from our discussion
|
| Comment by Bernard Gorman [ 13/Apr/23 ] |
|
sebastien.mendez@mongodb.com, sending this to you for now to find an assignee on your team. Per the conversation above, it seems like we can just broadcast for now and refine the targeting later. We may have to deal with a couple of JS test failures as a result, but we can likely just blocklist those. |
| Comment by Janna Golden [ 10/Apr/23 ] |
|
bernard.gorman@mongodb.com - ah so we will need these commands in the embedded router layer by 8.0 due to the "all clusters sharded in 8.0" plan laid out by the invisible sharding init. In 8.0, the embedded router layer in Serverless will very likely just be the same as the embedded router layer in dedicated clusters - today mongoq technically exists, though it actually just links in almost the entirety of mongos's link graph (the plan was always for mongoq to be built on top of mongos, and perhaps we'll actually re-evaluate whether mongos and mongoq should truly be different processes). In any case, in this ticket I think we can add the commands to the "embedded router layer"'s commands library, and fail the command if it's run without multitenancySupport enbaled - that way we ensure it isn't run in a dedicated setting. We don't yet have a "tenant shard targeting" protocol, so for now you could probably broadcast the request, including the tenantId sent in the original request. We can always change the targeting behavior once we've implemented "tenant shard targeting". |
| Comment by Bernard Gorman [ 07/Apr/23 ] |
|
steven.vannelli@mongodb.com, janna.golden@mongodb.com: sure, we can try to schedule this ticket - but I thought we'd have to wait until more work has been completed on the mongoq roadmap before it's feasible to do so. Is my understanding wrong? Is it actually feasible to do this work now? |
| Comment by Steven Vannelli [ 07/Apr/23 ] |
|
The Server Serverless team would like to have this ticket completed by 8.0 We have some time but figured we'd put it up for discussion for FY24 Q2 if the team has time. No harm in doing it early We might be able to place this in PM-3278 if folks agree with that. |