[SERVER-44921] 4.2: $changeStream cannot run with a readConcern other than 'majority' Created: 03/Dec/19 Updated: 27/Oct/23 Resolved: 13/Jan/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | 4.2.1 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Michael Smith | Assignee: | William Schultz (Inactive) |
| Resolution: | Works as Designed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Sprint: | Repl 2020-01-27 |
| Participants: |
| Description |
|
In a 3-node replica set with server 4.2.1 and featureCompatibilityVersion 4.2, I tried turning off majority read concern in my client and got:
The docs say:
That makes me think change streams should work in a cluster that has majority read concern enabled, but where the client chooses not to use it. |
| Comments |
| Comment by William Schultz (Inactive) [ 13/Jan/20 ] |
|
Ok, great. mzs I am going to close this ticket if you have no further questions. |
| Comment by Michael Smith [ 10/Jan/20 ] |
|
Yes, that's clear, thanks! |
| Comment by William Schultz (Inactive) [ 09/Jan/20 ] |
|
mzs The enableMajorityReadConcern flag is a per server (mongod process) parameter. If it is set to "true", then yes, it can cause cache pressure issues on that node if secondaries are lagging too far behind. If it is set to "false", however, cache pressure issues should not be a problem even if secondaries are lagging since we will not maintain history back to the majority commit point. Normally, if enableMajorityReadConcern:false, reads with "majority" read concern level will fail, since the server does not have the proper mechanisms enabled to satisfy them. For change streams, however, if you specify "majority" read concern level and run them against a 4.2 server with enableMajorityReadConcern:false, we have implemented an alternate mechanism for satisfying the "majority" guarantees required by change streams. This mechanism does not lead to cache pressure issues on the server. It was only implemented for change streams which is why they should succeed when you run them with "majority" read concern against a 4.2 server with enableMajorityReadConcern:false. Although it may seem counterintuitive that a "majority" read concern operation succeeds against a node with enableMajorityReadConcern:false, we made that choice to keep the external API the same so that existing users of change streams against nodes with enableMajorityReadConcern:true would not have to change their applications if they switched to using enableMajorityReadConcern:false. Does that clarify things for you? |
| Comment by Michael Smith [ 04/Dec/19 ] |
|
I understand enabling majority read concern at the server level can cause cache pressure when secondaries are lagging because the server needs to keep oplog entries in cache until they're majority-acknowledged. Would using a change stream with majority read concern in one small collection, cause bulk writes to an unrelated collection to get stuck in the cache when they otherwise wouldn't? ( |
| Comment by William Schultz (Inactive) [ 04/Dec/19 ] |
|
As mentioned in this docs section, even if any node in a cluster has enableMajorityReadConcern=false, change streams are still supported. When trying to use a change stream, the client still needs to specify read concern level "majority", though, since change streams fundamentally must only see "majority" committed data. One way to view it is that "majority" read concern level is still supported even if enableMajorityReadConcern=false, but only for change streams specifically. I can take a look over the rest of the documentation to see if we can make this fact more clear. |
| Comment by Michael Smith [ 04/Dec/19 ] |
|
I restarted my cluster with replication.enableMajorityReadConcern: false and got the same error.
But, with replication.enableMajorityReadConcern: false, if I set my $changeStream client to use majority read concern (even though it's disabled), it works. So it seems $changeStream wants you to ask for majority read concern, even if (as I understand it) the cluster is configured not to provide majority reads. |