[SERVER-37837] Possible for TransactionParticipant to never get cleaned up if no write happened Created: 30/Oct/18  Updated: 29/Oct/23  Resolved: 17/May/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.1.4
Fix Version/s: 4.1.12, 4.0.13

Type: Task Priority: Major - P3
Reporter: Randolph Tan Assignee: Kaloian Manassiev
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-40772 Prepare LogicalSessionCache for shutdown Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Sharding 2018-12-31, Sharding 2019-03-25, Sharding 2019-05-06, Sharding 2019-05-20
Participants:
Case:

 Description   

If a TransactionParticipant was created but never successfully persisted any changes to config.transactions, it can stay alive indefinitely. The most likely case for this to occur is when session is exclusively used for read only transactions. This is because the transaction reaper only looks at the config.transactions collection when deciding which transactions to reap.



 Comments   
Comment by Kaloian Manassiev [ 04/Oct/19 ]

This relates to retryable writes as well. For example, if the session of a retryable write which resulted in a no-op (didn't generate an oplog entry) was abandoned, the in-memory session state will not be cleaned-up.

Comment by Githook User [ 28/Sep/19 ]

Author:

{'username': 'kaloianm', 'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev'}

Message: SERVER-37837 Examine and reap sessions from the SessionsCatalog
Branch: v4.0
https://github.com/mongodb/mongo/commit/3e4cddb2866bcbb85195028d4d323f9300d24fb1

Comment by Githook User [ 27/Sep/19 ]

Author:

{'name': 'Kaloian Manassiev', 'username': 'kaloianm', 'email': 'kaloian.manassiev@mongodb.com'}

Message: SERVER-37837 Get rid of TransactionReaper

Part 1:

This change gets rid of the TransactionReaper's usage of the
ReplicationCoordinator for checking whether it is primary or not and
makes the LogicalSessionCache joinable on shutdown.

It also removes the TransactionReaper's grouping per-shard optimization
and moves it all under SessionCollectionSharded.

(cherry picked from commit 2791817876636c0cfd60d867f31c7a83cf3f18c1)

Part 2:

This change folds the TransactionReaper's single function to be part of
the SessionCatalogs on MongoD and MongoS, which are the subsystems
responsible for managing transactions.

(cherry picked from commit 94f269a1c6053824c4dabc50e8c9219b80a6a1b5)
(cherry picked from commit 7e6a80789cd74f9b533065f57afb5c9221eea1e7)
Branch: v4.0
https://github.com/mongodb/mongo/commit/2e165002e4b434e5713d8b7dff8d46151edff85d

Comment by Githook User [ 05/Sep/19 ]

Author:

{'name': 'Kaloian Manassiev', 'username': 'kaloianm', 'email': 'kaloian.manassiev@mongodb.com'}

Message: SERVER-37837 Move `config.transactions` manipulation out of SessionsCollection

(cherry picked from commit dabbf059e6f96edb4898b42d532d460efd2510f2)
(cherry picked from commit 2b40ec0f649def6e120b78510e8b008a43852a09)
Branch: v4.0
https://github.com/mongodb/mongo/commit/273ed84bc7eb6cc0e38eca16088fc1a454f1cb28

Comment by Githook User [ 17/May/19 ]

Author:

{'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com', 'username': 'kaloianm'}

Message: SERVER-37837 Examine and reap sessions from the SessionsCatalog

This change makes the logical sessions cache query and reap sessions,
which are possibly only in-memory on the SessionsCatalog.
Branch: master
https://github.com/mongodb/mongo/commit/aa9f6a202e0709adf14046cb27504864adaf732b

Comment by Githook User [ 09/May/19 ]

Author:

{'name': 'Benety Goh', 'username': 'benety', 'email': 'benety@mongodb.com'}

Message: SERVER-37837 fix mac os x compile
Branch: master
https://github.com/mongodb/mongo/commit/7e6a80789cd74f9b533065f57afb5c9221eea1e7

Comment by Githook User [ 09/May/19 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-37837 Get rid of TransactionReaper (Part 2)

This change folds the TransactionReaper's single function to be part of
the SessionCatalogs on MongoD and MongoS, which are the subsystems
responsible for managing transactions.
Branch: master
https://github.com/mongodb/mongo/commit/94f269a1c6053824c4dabc50e8c9219b80a6a1b5

Comment by Githook User [ 09/May/19 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-37837 Get rid of TransactionReaper (Part 1)

This change gets rid of the TransactionReaper's usage of the
ReplicationCoordinator for checking whether it is primary or not and
makes the LogicalSessionCache joinable on shutdown.

It also removes the TransactionReaper's grouping per-shard optimization
and moves it all under SessionCollectionSharded.
Branch: master
https://github.com/mongodb/mongo/commit/2791817876636c0cfd60d867f31c7a83cf3f18c1

Comment by Githook User [ 07/May/19 ]

Author:

{'name': 'Kaloian Manassiev', 'username': 'kaloianm', 'email': 'kaloian.manassiev@mongodb.com'}

Message: SERVER-37837 Use unique_ptr instead of shared_ptr for the SessionCatalog map
Branch: master
https://github.com/mongodb/mongo/commit/74d624eb406121e530330d0920cebc61d9aeae22

Comment by Githook User [ 02/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-37837 Fix compile
Branch: master
https://github.com/mongodb/mongo/commit/2b40ec0f649def6e120b78510e8b008a43852a09

Comment by Githook User [ 02/May/19 ]

Author:

{'name': 'Gregory Wlodarek', 'username': 'GWlodarek', 'email': 'gregory.wlodarek@mongodb.com'}

Message: SERVER-37837 Fix lint
Branch: master
https://github.com/mongodb/mongo/commit/455cde58a22f3d384b5d27202a64d3fabaa4d8ec

Comment by Githook User [ 02/May/19 ]

Author:

{'email': 'kaloian.manassiev@mongodb.com', 'name': 'Kaloian Manassiev', 'username': 'kaloianm'}

Message: SERVER-37837 Move `config.transactions` manipulation out of SessionsCollection
Branch: master
https://github.com/mongodb/mongo/commit/dabbf059e6f96edb4898b42d532d460efd2510f2

Comment by Gregory McKeon (Inactive) [ 09/Nov/18 ]

This also applies for TransactionRouter.

Comment by Kaloian Manassiev [ 01/Nov/18 ]

This applies to 4.0 as well, right?

And now to think of it - is it a problem in 3.6 as well, isn't it? If a no-op write executes, it is not going to write anything to config.transactions, but the in-memory object will remain. Meaning if the session is abandoned it will forever remain in-memory.

Comment by Randolph Tan [ 31/Oct/18 ]

After talking with Siyuan offline, realized that this is easier to hit than originally thought, updated description to mention read only transactions.

Comment by Siyuan Zhou [ 31/Oct/18 ]

The session reaper is different from transaction reaper. Session reaper looks up the session table and destroys the expired sessions. Since transaction participant is a decoration on session, it should be destroyed then.

Comment by Randolph Tan [ 31/Oct/18 ]

siyuan.zhou The transaction reaper only looks at config.transactions, so if there's no corresponding entry in there, then it doesn't exist from the point of view of the reaper.

Comment by Siyuan Zhou [ 30/Oct/18 ]

Does the session reaper kill this transaction participant?

Generated at Thu Feb 08 04:47:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.