[SERVER-37669] Allow change streams to project out the resume token Created: 19/Oct/18  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Kristofer Brandow (Inactive) Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: change-streams-improvements
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-37786 Reject change streams which modify or... Closed
Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Participants:

 Description   

In SERVER-37786 we banned change stream pipeline which modified or projected-out the _id because the _id is important for the driver protocol. There's no real reason we couldn't allow this, but doing so would require a different protocol, described below.

Original Title

Add Resume Tokens To Cursor Body

Original Description

Resume tokens are currently returned as part of the documents from a change stream. This is useful because it allows users to retrieve the resume token and provide it as an argument later, but it is easy to accidentally remove it via $project or $replaceRoot. In the case of a $replaceRoot the user might replace the _id field with another type and drivers would attempt to use that as a resume token, which would fail at a much later point in time.

To avoid the problem altogether, an array of resume tokens should be provided at the top level of the cursor body, alongside the firstBatch and nextBatch fields. To enable users to retrieve the resume token and to prevent breaking current drivers, the resume token should still be available on each of the change stream documents.

This feature could potentially be implemented alongside SPEC-1122.



 Comments   
Comment by Charlie Swanson [ 01/Apr/19 ]

ian.whalen I think this ticket still describes something we may consider doing in the future, but is not in any plan. SERVER-37786 was implemented because the scenario of projecting out the resume token had never worked as you might expect it to. This ticket describes a protocol change that would enable that use case. I've re-titled the ticket to be about the use-case, and I propose we leave it open and on the backlog.

Comment by Ian Whalen (Inactive) [ 01/Apr/19 ]

charlie.swanson is this still needed given the resolution of SERVER-37786?

Comment by Charlie Swanson [ 26/Oct/18 ]

Putting this back on the query backlog for triage again.

kris.brandow convinced me this is still a legitimate improvement to the change stream protocol that may be worth doing. Should we do so, it will certainly have drivers implication though, so I think perhaps we should convert this to an epic.

Comment by Charlie Swanson [ 26/Oct/18 ]

charlie.swanson to repurpose this ticket into a request for the server to reject change streams which do not preserve the _id.

Generated at Thu Feb 08 04:46:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.