[SERVER-37538] Feature that limits size or query open count of ChangeStream Created: 10/Oct/18  Updated: 09/Nov/18  Resolved: 09/Nov/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Minsu Kim Assignee: Asya Kamsky
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Requirements about ChangeStream management:

  1. limit count of ChangeStream for specific database/collection
  2. query open count of ChangeStream for specific database/collection

 

Many services might want to have high availability to process ChangeStream and avoid duplicates simultaneously. So they want to maintain only one valid ChangeStream for topic (db/collection) in several application instances.

But I know currently it has to be implemented in applicatoin side.

 



 Comments   
Comment by Asya Kamsky [ 09/Nov/18 ]

You can use existing commands to see how many change streams exist for each db/collection and you can use that information to terminate excess ones.  I'm closing this ticket as I don't think we want to add a server-side limit for the number of change streams.

 

Comment by Minsu Kim [ 23/Oct/18 ]

@Asya Kamsky

Yes, what you said is right.

But MongoDB already provides second feature, I think first feature might be just optional.

 

Thanks

Comment by Asya Kamsky [ 22/Oct/18 ]

iqjumps

Can you clarify what each of these mean?

1.  limit count of ChangeStream for specific database/collection
Do you mean that somewhere on the server that you would be able to set a limit (say 5) and if there is a sixth ChangeStream request for particular collection it would fail with an error that there are already 5?

2. query open count of ChangeStream for specific database/collection
You can already do this using $currentOp aggregation stage and some filtering and grouping after.
Here is an example from the shell:

db.aggregate([{$currentOp:{}},{$match:{"originatingCommand.pipeline.0.$changeStream":{$exists:true}}},{$group:{_id:"$ns", count:{$sum:1}}}]).pretty()
{ "_id" : "test.newcoll", "count" : 1 }

If this is adequate for tracking how many change streams there are against which namespaces, then just clarify what 1 (limiting) should be.

Generated at Thu Feb 08 04:46:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.