[SERVER-74964] Pass namespace to LastOpFixer for each operation Created: 16/Mar/23  Updated: 29/Oct/23  Resolved: 07/Apr/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.0.0-rc0

Type: Task Priority: Major - P3
Reporter: Sean Zimmerman Assignee: Kaitlin Mahar
Resolution: Fixed Votes: 0
Labels: milestone-1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-72808 Support and test bulkWrite with write... Closed
Assigned Teams:
Replication
Backwards Compatibility: Fully Compatible
Sprint: Repl 2023-04-17
Participants:

 Description   

LastOpFixer currently can only be constructed with a Namespace. For bulkWrite we want to be able to construct it without a Namespace so we only have to use one object for the entire write which can take place across multiple namespaces. For bulkWrite we would like to move this instantiation into performWrites

 

A couple ways this could be done

make a ctor that allows us to set isLocalDB directly. Unless all namespaces used in a bulkWrite are local, otherwise isLocalDB should be false and we should set lastOp correctly for no-op writes on replicated namespaces

 

let startingOp() take in a namespace/bool indicating whether it is a local write, if it is, then we make the startingOp() and finishedOpSuccessfully() pair no-op for that particular write

 



 Comments   
Comment by Kaitlin Mahar [ 12/Apr/23 ]

I realized while working on SERVER-75798 that the tests I added as part of this ticket were not sufficient to actually confirm we actually wait for write concern correctly for batched inserts via bulkWrite.

The way that the noop_writes_wait_for_write_concern.js tests work is:

  1. Create a 3 node replica set
  2. Stop one of the secondaries
  3. With one connection, run setup code
  4. With a fresh connection (where the client would have no last optime to set to begin with), run a command containing a no-op write using write concern w:3 and wtimeout: 1000
  5. Expect that the command returns a write concern error

For a command containing a single write operation, step 5 proves that we set the client's last optime even when we no-op, and that we then wait for all nodes to catch up to that optime. If we did not do both of those things, then we wouldn't receive a write a concern error.

However, for a batched write this assertion is not strong enough to prove that we correctly advance the client's last optime all the way to the latest oplog entry's optime. To illustrate with an example:

Consider a batch of two inserts where the first insert op1 is successful and the second insert op2 is a no-op, e.g. due to a DuplicateKeyError. The correct behavior would be for op1 to advance the client last optime to its own optime, and for op2 to advance the client's last optime to the latest oplog entry's optime, and for us to then wait for the latest optime to replicate.

The issue lies in that even if we only waited for op1's optime to replicate, and not the latest optime, we would still receive a write concern error, since the node has been stopped all along and neither of the writes will replicate. So the write concern error does not prove we "fixed" the optime for the no-op.

As part of SERVER-75798 I will write a new test that is similar but allows us to actually prove we update the optime for the no-op, and I will switch to running the batched insert test cases I wrote here in that way.

Comment by Githook User [ 07/Apr/23 ]

Author:

{'name': 'Kaitlin Mahar', 'email': 'kaitlin.mahar@mongodb.com', 'username': 'kmahar'}

Message: SERVER-74964 Pass namespace to LastOpFixer for each operation
Branch: master
https://github.com/mongodb/mongo/commit/759110fa276dc332c2dec26d71c7246f27ebc391

Comment by Kaitlin Mahar [ 06/Apr/23 ]

For future readers, what we decided to do here was the second solution, i.e. the namespace is now passed to startingOp() and we no longer accept the namespace in the constructor.

Generated at Thu Feb 08 06:28:59 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.