[SERVER-80862] Make config.transactions a clustered collection Created: 07/Sep/23  Updated: 31/Jan/24  Resolved: 23/Jan/24

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.3.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Matthew Russotto Assignee: James Bronsted
Resolution: Fixed Votes: 0
Labels: perf-8.0, perf-tiger, perf-tiger-handoff, perf-tiger-poc, perf-tiger-project-candidates, perf-tiger-q4
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-86043 Cluster config.transactions Closed
Related
related to SERVER-85423 CollectionScan::work() returns NEED_T... Closed
related to SERVER-82673 Create commands that reserialize conf... Open
related to SERVER-84073 Enable the ClusteredConfigTransaction... Open
Assigned Teams:
Service Arch
Backwards Compatibility: Fully Compatible
Sprint: Service Arch 2023-10-16, Service Arch 2023-10-30, Service Arch 2023-11-13, Service Arch 2023-11-27, Service Arch 2023-12-11, Service Arch 2023-12-25, Service Arch 2024-01-08, Service Arch 2024-01-22, Service Arch 2024-02-05
Participants:

 Description   

Original proposal:
Currently every time we do a retryable write, we find the _id index for the config.transactions table, look up the record ID we want to write in it, and then update the record. But we know the _id will always be the session ID which does not change for a given TransactionParticipant, so we can cache that record ID on the participant the first time we insert or look it up, and skip the index lookup on most writes.

We have to handle two exceptional cases

1) The record ID no longer exists, e.g. because the session got reaped yet re-used.

2) The record ID exists, but does not contain our _id value (the record ID got re-used). I don't know if this is possible with WiredTiger but I don't think we have formal guarantees against it.

In both cases we would just fall back to the slow path of doing a lookup and possibly an insert.

New proposal:
Make the config.transactions collection clustered. This is a cleaner implementation but comes with upgrade/downgrade considerations.



 Comments   
Comment by Githook User [ 23/Jan/24 ]

Author:

{'name': 'James Bronsted', 'email': '32047428+jpbronsted@users.noreply.github.com', 'username': 'jpbronsted'}

Message: SERVER-80862 change config.transactions to be a clustered collection (#16690)

GitOrigin-RevId: 671bae337ecc0d269153dd749c22e4f5abcb1f57
Branch: master
https://github.com/mongodb/mongo/commit/473009cc3c3d190a9baa70f87a714dbd176103b3

Comment by Matthew Russotto [ 11/Sep/23 ]

Yes, config.transactions is implicitly replicated.

Comment by Lingzhi Deng [ 08/Sep/23 ]

george.wangensteen@mongodb.com I think I missed the comment from matthew.russotto@mongodb.com in PERF-4646. Looks like we did a POC to cluster config.transactions there too and we saw similar gain. But as Matthew said, it will come with upgrade/downgrade implication which I think it shouldn't be too bad.

Comment by Jason Chan [ 07/Sep/23 ]

george.wangensteen@mongodb.commax.hirschhorn@mongodb.com, is this something we can add to PM-3326?

Generated at Thu Feb 08 06:44:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.