[SERVER-32895] Provide method for getting 'current resume token' for a collection Created: 25/Jan/18 Updated: 06/Dec/22 Resolved: 13/Aug/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | 3.6.0, 3.6.1, 3.6.2 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Robert Grimball | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Duplicate | Votes: | 2 |
| Labels: | changestreams | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query
|
||||||||
| Participants: | |||||||||
| Description |
|
Having a way to query the collection for the most recent resume token is extremely useful for providing that on server startup to clients who are being sent initial data and then relying on the resume tokens to keep them 'up to date' and in 'real time', even after a disconnect. Scenario: ServerA makes rare changes to CollectionA via some mechanism. ServerB provides an endpoint which allows BrowserB to get all documents from CollectionA and keep up to date with any changes(via change streams) made to the collection(whether via other Browsers interacting with ServerB, or the occasional update ServerA makes). However, no update has been made yet to CollectionA, so when the initial CollectionA is queried for all docs, BrowserB gets it, but ServerB does not have a resume token to send along after it is done(no updates have been made to CollectionA yet). ServerB crashes, disconnecting from BrowserB BrowserB is trying to reconnect to ServerB ServerA makes an update to CollectionA ServerB starts up again, no updates since start so no resume token for CollectionA BrowserB reconnects, but has no resume token. At this point, since ServerB was unable to send to BrowserB the current resume token for CollectionA right after initially sending the collection, nobody knows where to resume from, and the entire collection will have to be resent, rather than taking advantage of the resumability of change streams. If ServerB was able to ask for CollectionA's resume token and pass that along with the end of streaming the documents intially, then BrowserB would simply reconnect to the newly up server, providing it's resume token, and catch back up from where it left off, getting the update from ServerA and moving on along. The only way to hack around this that I know of is to force an update to a document on server up to get a resume token so you can send that along to anyone who requests streaming. Providing the capability to simply ask for what was the last resume token for a given collection prevents the need for this ugly hack. |
| Comments |
| Comment by Robert Fox [ 14/Aug/19 ] |
|
Thank you @"Charlie Swanson" I've also now discovered "startAtOperationTime" parameter in Collection.watch(), but one question worth asking... how can I get the server time from MongoDB? I don't want to use client generated timestamps. |
| Comment by Charlie Swanson [ 13/Aug/19 ] |
|
Great. I'm going to close this issue as a duplicate. Please leave a comment if you believe this is incorrect and explain the difference if so. |
| Comment by Bernard Gorman [ 13/Aug/19 ] |
|
charlie.swanson: yes, I believe the high-water-mark feature completed under |
| Comment by Charlie Swanson [ 12/Aug/19 ] |
|
Forgive me as it is not the most legible description, but I think that we solved this in Does that look like it would work for your use case fox.bob@gmail.com and/or rgrimball? bernard.gorman do you think we should close this ticket as a duplicate? |
| Comment by Robert Fox [ 12/Aug/19 ] |
|
Change streams are important for Mongo DB! I can't really use them without this functionality... the ability to resume a stream without first seeing a change. If you don't incorporate this basic stuff you're basically telling people: go use Kafka. |
| Comment by Kelsey Schubert [ 26/Jan/18 ] |
|
Hi rgrimball, Thanks for the improvement request. We've put this ticket on our backlog, and we'll update the fixversion if it is scheduled for a release. Kind regards, |