[SERVER-38048] Allow an update query to be run always at the end of the session Created: 09/Nov/18  Updated: 09/Nov/18  Resolved: 09/Nov/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: NOVALUE Mitar Assignee: Danny Hatcher (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

I would like to ask for a feature which would allow me to specify an update query to be always run at the end of the session. Even if the client dies/disconnects.

I have a schema design where I have two collections, Document and Step. Each Document can have multiple Steps. I use two collections because size and number of Steps belonging to one Document can grow over the 16 MB limit so I cannot include them inside one Document.

In most operations I just add more Step documents. But some operations require changing many Step documents at once and during that time no new documents should be added for a particular Document. (Imagine like rewriting git history.) For this, I have a field on Document which serves as a lock. The logic is as follows:

  • Set lock on Document.
  • Add a Step document, or modify multiple Step documents.
  • Release lock on Document.

The problem is that if the client dies/disconnects before releasing a lock, I have a problem. It seems MongoDB transactions would not help me here with an overall problem. They can help with making modification of multiple Step documents all at once or none. But during transaction other Step documents could still be added. Which I would like to prevent.

So I think that what would solve my problem is that I could set an update query to be run at the end of the session. Then I could do:

  • Start session.
  • Set lock on Document.
  • Set release lock query on Document to be run at the end of session.
  • Start transaction.
  • Modify multiple Step documents.
  • Commit transaction.
  • End session.

 



 Comments   
Comment by NOVALUE Mitar [ 09/Nov/18 ]

I see. Thanks. Feel free to close the issue then.

Comment by Andy Schwerin [ 09/Nov/18 ]

You could keep the "lock" in a separate document, instead of in Document itself. That would allow concurrently updates to Document and to Steps associated with the Document. If you're using transactions anyhow, that decomposition should be safe. In any event, with the feature you requested or without it, your application's concurrency control is complex and you'll want to think carefully about whether its value is worth the programming complexity.

As to your requested feature, MongoDB may not know for a very long time that a session is over, and no particular node can ever be certain that the session is over. Sessions aren't bound to individual TCP connections, or even particular client/host ip pairs. As such, I don't think your request is practical.

Comment by NOVALUE Mitar [ 09/Nov/18 ]

Sure, but this then prevents any modification to Document while Steps are being updated. I would hope to be able to avoid that.

Comment by Danny Hatcher (Inactive) [ 09/Nov/18 ]

Hello Mitar,

I believe you should be able to use transactions for the entire "locking/unlocking" process.

1. Start session
2. Start transaction
3. Update the Document (this will prevent anything from updating the Document until the transaction either commits or aborts)
4. Modify multiple Step documents
5. Update the Document again to set it to its original state
6. Commit transaction
7. End session

Here is a quick example:

replset:PRIMARY> db.document.findOne()
{ "_id" : 1, "lock" : false }
replset:PRIMARY> db.step.find().pretty()
{ "_id" : 1, "docid" : 1 }
{ "_id" : 2, "docid" : 1 }
{ "_id" : 3, "docid" : 1 }
{ "_id" : 4, "docid" : 1 }
replset:PRIMARY> var session = db.getMongo().startSession();
replset:PRIMARY> session
session { "id" : UUID("941f9ba3-40be-4a1f-b89d-0548ea5c74d7") }
replset:PRIMARY> db = session.getDatabase(db.getName());
test
replset:PRIMARY> session.startTransaction()
replset:PRIMARY> db.document.update({_id: 1}, {lock: true})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
replset:PRIMARY> db.step.update({_id: 1}, {test: "success"})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
replset:PRIMARY> db.document.update({_id: 1}, {lock: false})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
replset:PRIMARY> session.commitTransaction()

On another connection I ran the following after I updated the document inside the transaction:

replset:PRIMARY> db.document.update({_id:1}, {a:1})

This update hung until the transaction was committed. If the client had lost the connection, the transaction would have timed out and the internal write lock would have been released.

Do you think the above will not work for your use case?

Generated at Thu Feb 08 04:47:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.