[CXX-2435] Why does change stream end when more logs exist Created: 31/Jan/22 Updated: 27/Oct/23 Resolved: 03/Feb/22 |
|
| Status: | Closed |
| Project: | C++ Driver |
| Component/s: | API |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Unknown |
| Reporter: | Oded Raiches | Assignee: | Kevin Albertson |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | changestreams | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Description |
|
Hi, I'm using change streams to gather statistics about my operations (need to take some info from each insert/update) - so I need it to act as a listener without closing. I inserted 1,000,000 documents into a collection and on the change stream side I am using the start_at_operation_time() option to gather all the required info. I noticed that the change stream closes randomly before all 1M objects are viewed (using a counter in the loop). Also, I noticed that when I configure the max_await_time() to 5000 ms then it works fine and all documents are captured before it closes. The behavior I need is to have the change-stream as a listener that waits for changes all the time. Is there a way to have this loop blocking? meaning that it will always wait for the next incoming change without exiting the loop. I saw that someone once issued a "bug" about it, but this is the behavior I want to implement: From some POCs that I made with Golang and python driver, it looks like they are always blocking and waiting for the next change.
mongocxx driver version: 3.6.6 mongoc driver version: 1.20.1
|
| Comments |
| Comment by Kevin Albertson [ 03/Feb/22 ] | |||||||||||||||||||||||||
Great to hear.
operator++() and begin() are blocking operations. They may block for up to max_await_time waiting for a possible notification. I think the documentation could clarify what is meant. I created | |||||||||||||||||||||||||
| Comment by Oded Raiches [ 02/Feb/22 ] | |||||||||||||||||||||||||
|
Works! thanks a lot. On the documentation for operator++() and begin() it says that they are blocking. Does this mean that there is still a bug that should be fixed here? please answer whether this is a problem in mongocxx or the mongoc driver, or just the documentation: | |||||||||||||||||||||||||
| Comment by Kevin Albertson [ 02/Feb/22 ] | |||||||||||||||||||||||||
|
Incrementing the iterator with cs_iter++ invalidates the bsoncxx::document::view returned by *cs_iter. Putting the access to event inside the loop should fix that segfault:
| |||||||||||||||||||||||||
| Comment by Oded Raiches [ 02/Feb/22 ] | |||||||||||||||||||||||||
|
Hi Kevin, When using this approach, I get a segfault after 11197 loops on every attempt (when trying to access the event). A bit of what I was trying to do:
| |||||||||||||||||||||||||
| Comment by Kevin Albertson [ 01/Feb/22 ] | |||||||||||||||||||||||||
|
Hello oraiches@zadarastorage.com,
The iterator returned by mongocxx::change_stream::begin() is exhausted after one attempt to poll for new events (with the getMore command) from the server returns no results. To continue iterating a change stream until the next event or error, the current API requires calling mongocxx::change_stream::begin() again to continue iteration.
| |||||||||||||||||||||||||
| Comment by Kevin Albertson [ 31/Jan/22 ] | |||||||||||||||||||||||||
|
Hello oraiches@zadarastorage.com, thank you for the report. We will look into this soon. |