[SERVER-72718] Not able to identify the transaction boundaries using ChangeStream Created: 11/Jan/23  Updated: 19/Jan/23  Resolved: 18/Jan/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Kiruphasankaran Nataraj Assignee: Yuan Fang
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-72885 Are change events of a single transac... Closed
Related
related to SERVER-40437 [Change Streams] Allow to recognize l... Backlog
is related to SERVER-72885 Are change events of a single transac... Closed
Participants:

 Description   

The MongoDB documentation (https://www.mongodb.com/docs/manual/reference/change-events/) specifies that for a change event, the transaction associated with that event can be uniquely identified by the lsid and txnNumber fields in the ChangeStream document.

The problem during change data capture using ChangeStream is that I'm not able to determine whether the transaction has ended or not unless and otherwise, the next transaction comes in (different txnNumber) or a non-transactional database operation (lsid and txnNumber are null using the Java driver) is performed. Is there any way to determine if the current transaction has ended or not from the ChangeStream without needing another consecutive operation?



 Comments   
Comment by Kiruphasankaran Nataraj [ 19/Jan/23 ]

Hi Yuan,

Thanks for your response! One more final clarification. Are the change events of a transaction always guaranteed to appear in the order in which they are executed? For example, if two concurrent transactions are being executed, the below sequence shows the order of the operations being maintained even if the transactions themselves are not grouped together.

  • txn1      -       operation1
  • txn1      -       operation2
  • txn2     -       operation1
  • txn1      -       operation3
  • txn1      -       operation4
  • txn2     -       operation2
  • txn2     -       operation3
  • txn2     -       operation4

Is there a possibility of the change events appearing as:

  • txn1      -       operation1
  • txn1      -       operation2
  • txn2     -       operation1
  • txn1      -       operation4
  • txn1      -       operation3
  • txn2     -       operation4
  • txn2     -       operation3
  • txn2     -       operation2
Comment by Yuan Fang [ 17/Jan/23 ]

Hi kirupha2000@gmail.com,

Thank you for bringing this to our attention. As you have noted, the feature that specifically identifies the last change event associated with a transaction is not currently supported. It seems that the issue (SERVER-40437) is backlogged at the moment.

In response to your question:

is it guaranteed that the change events of a single transaction are always grouped together even if multiple transactions are executed from multiple clients concurrently?

It is not guaranteed that the change events of a MongoDB transaction will always be grouped together. Change events are generated by Oplog and the order of events may vary depending on the concurrent nature of the operation. I hope this explanation is helpful.

If you have any further questions, we highly encourage you to first seek assistance from our community by posting on  MongoDB Developer Community Forums. If the discussion there leads you to suspect a bug in the MongoDB server, then we'd want to investigate it as a possible bug here in the SERVER project.

Regards,
Yuan

 

Comment by Kiruphasankaran Nataraj [ 13/Jan/23 ]

Based on the comments from https://jira.mongodb.org/browse/SERVER-40437, it looks like MongoDB doesn't currently support determining the boundaries of a transaction. Please correct me if I'm wrong. Also, is it guaranteed that the change events of a single transaction are always grouped together even if multiple transactions are executed from multiple clients concurrently?

Generated at Thu Feb 08 06:22:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.