[SERVER-42114] Allow single RS transactions to run on the config database excluding config.transactions Created: 08/Jul/19  Updated: 29/Oct/23  Resolved: 01/Aug/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.3.1

Type: Task Priority: Major - P3
Reporter: Jack Mulrow Assignee: Blake Oler
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Gantt Dependency
has to be done before SERVER-42143 Convert refineCollectionShardKey meta... Closed
Related
related to SERVER-28756 Remove TODO SERVER-28756 comment and ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2019-07-29, Sharding 2019-08-12
Participants:

 Description   

Currently all transactions are disallowed on the config database. To allow refineCollectionShardKey to use a single replica set transaction on the config server primary that updates entries in config.collections, config.chunks, and config.tags, this restriction will need to be relaxed to only disallow transactions on config.transactions.

This restriction will not be relaxed on mongos because currently transaction numbers are not propagated by mongos for writes on the config database because of its custom CSRS write retry logic (which will be removed by SERVER-28756).

Proposed implementation:



 Comments   
Comment by Githook User [ 01/Aug/19 ]

Author:

{'name': 'Blake Oler', 'username': 'BlakeIsBlake', 'email': 'blake.oler@mongodb.com'}

Message: SERVER-42114 Allow single RS transactions to run on the config database in sharded clusters
Branch: master
https://github.com/mongodb/mongo/commit/927883be3ec0af1f1a8ac65176f39696a31b4962

Comment by Siyuan Zhou [ 11/Jul/19 ]

LTGM. Thanks for posting the list!

Comment by Jack Mulrow [ 11/Jul/19 ]

siyuan.zhou, like we talked about offline, I ran listCollections on the config database on shards, the config server, and a standalone RS to get a sense for the risk of allowing users to run transactions on config and here are the collections I found (note this may have missed some transient collections):

On the config server replica set:

  • "config.actionlog"
  • "config.changelog"
  • "config.chunks"
  • "config.collections"
  • "config.databases"
  • "config.lockpings"
  • "config.locks"
  • "config.migrations"
  • "config.mongos"
  • "config.settings"
  • "config.shards"
  • "config.tags"
  • "config.transactions"
  • "config.version"

On a replica set shard:

  • "config.cache.chunks.config.system.sessions"
  • "config.cache.chunks.db.coll"
    • Note for every sharded collection a shard owns chunks for there will be a "config.cache.chunks.<namespace>" collection
  • "config.cache.collections"
  • "config.cache.databases"
  • "config.system.sessions,
  • "config.transactions"
  • "config.transaction_coordinators"

On a standalone replica set:

  • "config.system.sessions"
  • "config.transactions"

As far as I can tell, these are all collections that can already be written to normally outside of a transaction except config.transactions (which can only be written to without a session and on a standalone node), so I don't think there's too much risk allowing them to be written to in a transaction.

Comment by Kaloian Manassiev [ 10/Jul/19 ]

jack.mulrow, I have no objections with continuing to disallow transactions and retryable writes against the config database through mongos.

Comment by Jack Mulrow [ 10/Jul/19 ]

I was planning on allowing all users to run transactions on config collections (other than config.transactions) since I couldn't think of a specific reason why not to and it seems like a useful feature. We only need this capacity for internal operations though, so I'm fine restricting it to internal users if you prefer that.

Comment by Siyuan Zhou [ 09/Jul/19 ]

I cannot think of other collections in "config" database off the top of my head. Replication uses "local" database in most cases.

Are you going to achieve this with access control? If we only give internal code this permission, I'd feel more confident with the implementation.

Comment by Jack Mulrow [ 08/Jul/19 ]

siyuan.zhou, do you know if there are any collections in config that we should disallow transactions on other than config.transactions?

kaloian.manassiev, are you OK with my proposal to keep disallowing transaction commands on config on mongos? We can also fix the mongos write path to not drop transaction numbers so we can lift the restriction there too, which might be worth doing for feature parity between sharded and non-sharded transactions, but that isn't required for the refine the shard key project.

Generated at Thu Feb 08 04:59:37 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.