[SERVER-40437] [Change Streams] Allow to recognize last change of transaction Created: 02/Apr/19  Updated: 13/Jan/23

Status: Backlog
Project: Core Server
Component/s: Querying
Affects Version/s: 4.0.8
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Stefan Holzapfel Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 4
Labels: change-streams-improvements
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-72718 Not able to identify the transaction ... Closed
Assigned Teams:
Query Execution
Participants:

 Description   

I'm using Mongo change streams to send database changes in realtime from my node backend to an angular webapplication.

When I mutate a lot of data rows using a transaction, I want to send those changes to the client as one accumulated change and not one by one.

To achive that, I queue new changes if they are part of a transaction.

And here starts my problem:

How to identify the last change event of a transaction? At the moment I set a timeout and wait if another change with the same lsid arrives within the timeout. If not I consider the transaction complete.

Aside from being a very ugly hack, this has two severe downside:

  • If the timeout is too short, I can't rely on the transaction changes to arrive fast enough
  • The timeouts accumulate and lead (in case of transactions affecting a lot of documents) to very high response times on the client side

I think the best solution would be to either set a flag on the change event (something like a "last" or "finished" boolean in addition to the lsid) or emit a specific event in the stream after a transaction's last change.

If anybody has already solved that issue in another fashion, please let me know.

Thanks very much!



 Comments   
Comment by Stefan Holzapfel [ 05/Apr/19 ]

Great, thx!

Comment by Eric Sedor [ 05/Apr/19 ]

Thanks for your request stefanholzapfel. We're assigning this ticket to the appropriate team to be evaluated against our currently planned work. Updates will be posted on this ticket as they happen.

Generated at Thu Feb 08 04:54:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.