[SERVER-23923] Agg $out writeConcern not enforced when batchSize=0 Created: 25/Apr/16  Updated: 05/Jun/19  Resolved: 03/Jun/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.1.14

Type: Bug Priority: Major - P3
Reporter: Mathias Stearn Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
is related to SERVER-41444 Improve WriteConcernOptions to differ... Backlog
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Query 2016-12-12, Query 2019-06-03, Query 2019-06-17
Participants:

 Description   

judah.schvimer's work to have writeConcern handled at the command processor won't work correctly with $out if the aggregate command returns a cursor with batchSize=0 because the writes are done in the getmore, not in the aggregate command



 Comments   
Comment by Githook User [ 03/Jun/19 ]

Author:

{'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}

Message: SERVER-23923 Track writeConcern through getMores
Branch: master
https://github.com/mongodb/mongo/commit/9bcb318c4c9f2c482413a9d85b7e7c99c0492653

Comment by Tess Avitabile (Inactive) [ 22/May/19 ]

That sounds reasonable to me. And in the CR, I requested an error check that will help us notice if we change getMore to accept a writeConcern (and then silently ignore it, since we override it using the one from the cursor).

Comment by Charlie Swanson [ 22/May/19 ]

"never" seems a little strong for my liking but here's my argument:

  • It is consistent with other options like readConcern and maxTimeMS: it is specified to the initial cursor-generating command and then applied to subsequent batches implicitly.
  • We don't have to think about the case where the writeConcern changes in the middle of iterating the cursor. If one getMore has a different writeConcern than the previous one.
  • It seems less complicated for drivers? Shooting in the dark here but it seems simpler to me on the surface.

It is interesting here that our cursor-generating write commands don't actually return anything ($out and $merge), so it's a little weird to think about. When reasoning about this in the abstract I'm imagining something like a cursor-style findAndModify with multi: true. In that case I think it would make sense to apply the writeConcern at the level of the command/operation, and not on each getMore. I think for the same reason each batch in an unordered bulk op needs to wait for write concern (I recall it being because there could be rollback of writes from a previous batch before you get to the end), we'd need to wait for writeConcern on each getMore. By this reasoning it's not crazy to specify a writeConcern to each getMore, but it seems more error-prone without any utility.

Who else has opinions here? schwerin?

Comment by Tess Avitabile (Inactive) [ 22/May/19 ]

If we are going with option 2, then I want to check that we are committed to never supporting writeConcern for getMore.

Comment by David Storch [ 15/Dec/16 ]

The drivers team plans to ensure that $out with a batchSize of 0 is disallowed, so we are not planning to do any server-side work on this issue at the moment. I've tried to summarize our discussion around this issue below.

It seems that we have two options, each with pros and cons:

  1. Make the getMore command accept a writeConcern.
  2. When an aggregation cursor is established, attach the writeConcern to the cursor and apply it on subsequent getMores.

Option #1 has a more natural implementation in the server, since it makes getMore no different from other commands which write. Option #2 would require us to have special writeConcern handling in the getMore path, which could result in duplicated logic to maintain. Option #1 also means that the exposed user interface is more flexible, since a client could in theory specify a different write concern for the initial agg command and the getMore. On the other hand, drivers might not have a natural way to expose this flexibility to the application, and, even if they wanted to do so, this could involve a lot of driver-side development work.

Generated at Thu Feb 08 04:04:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.