[SERVER-5951] find_try_modify and $collide operator to streamline two-phase commit Created: 29/May/12  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Querying, Write Ops
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Jason Voorhees Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: core, transactions
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Optimization
Participants:

 Description   

Let's say you're attacking an update with a two-phase commit, colliding on a timestamp field. The second phase uses find_and_modify to ensure the timestamp is the same before applying the update. So far so good. However, should the timestamp be unequal, find_and_modify will simply fail to find, returning no document. In order to take another crack at the update, it's probably necessary to query the record all over again to find out what changed, which is kind of a waste since mongo has (probably) already found the record once before disqualifying it.

An alternate proposal is find_try_modify, basically the opposite of find_and_modify. It returns a document only if the update fails, which should definitely be expressed as an exception by the drivers. For a nice example (in say, python) let's try to $set `resultField` based on data from `documentState` but only if `timestamp` hasn't changed. If it has, we want to look at `documentState` again. The server can assume we want `timestamp` too, since we set it as a collision field. Perhaps setting it as false in the return fields should be permitted, as it is with _id.

try:
 db.Records.find_try_modify(
      {_id:578923, $collide:{timestamp:123456789}}}, 
      {$set:{documentState:stateObject}},
      {documentState:true}
)
 except Collision as newState:
      stateObject = buildUpdate (newState['documentState'])
      try: # just one more attempt
      db.Records.find_try_modify(
          {_id:578923, $collide:{timestamp:newState['timestamp']}}, 
          {$set:{documentState:stateObject}},
          {documentState:true}
       )
       except Collision:
           sendFailureResponse()
 sendSuccessResponse()
 



 Comments   
Comment by Asya Kamsky [ 30/Jun/19 ]

Setting for triage to see if there is still a use case for adding this or if the ticket should be closed.

Comment by Asya Kamsky [ 30/Jun/19 ]

My thinking was that we should close this ticket given there are transactions available already.

Proposed flow (without using timestamp as the transaction machinery will let you know if there is a write conflict):

Start session
Start transaction
Get document
Make changes
Save document+Commit transaction

If you get a write conflict at the last step, you have to get the document again.

On the other hand, the read-modify-save pattern should not be necessary in 4.2 given that updates can now be expressions as aggregation pipeline with full access to all the fields of the document during the update.

So the given example:
> let's try to $set `resultField` based on data from `documentState` but only if `timestamp` hasn't changed.

can be done without two-phase-commit approach simply with:

// update could be find and modify
db.coll.update( {_id: 578923}, 
                           [ { $set: { resultField: "$documentState"} } ] 
)

"$documentState" can be any agg expression which references any field of the document, here I'm just setting resultField to whatever is in documentState but instead it could be used as a conditional/branch. See examples here: https://docs.mongodb.com/master/reference/command/update/index.html#update-command-example-agg

Comment by Eliot Horowitz (Inactive) [ 17/Jan/13 ]

Agree we should make this easier, not sure what the optimal syntax is

Comment by Jason Voorhees [ 29/May/12 ]

Um. Lemme try that again...

Example

try:
     db.Records.find_try_modify({_id:578923, $collide:{timestamp:123456789}}}, {$set:{documentState:stateObject}}, {documentState:true})
except Collision as newState:
     stateObject = buildUpdate (newState['documentState'])
     try: # just one more attempt
          db.Records.find_try_modify({_id:578923, $collide:{timestamp:newState['timestamp']}}, {$set:{documentState:stateObject}}, {documentState:true})
     except Collision:
          sendFailureResponse()
          return
sendSuccessResponse()

Generated at Thu Feb 08 03:10:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.