[SERVER-61748] dbCheck should not hold a strong database lock during batches Created: 26/Nov/21  Updated: 29/Oct/23  Resolved: 08/Dec/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.2.0, 5.0.6, 4.4.11

Type: Improvement Priority: Major - P3
Reporter: Josef Ahmad Assignee: Josef Ahmad
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
duplicates SERVER-43445 Remove 'replicatedSystemCollections' ... Closed
Related
related to SERVER-61765 Remove dbCheck dependency on Deferred... Backlog
related to SERVER-30932 dbCheck command violates lock orderin... Closed
related to SERVER-61877 Remove catalog consistency verificati... Closed
is related to SERVER-61754 dbCheck should not hold a strong coll... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.1, v5.0, v4.4
Sprint: Execution Team 2021-11-29, Execution Team 2021-12-13
Participants:

 Description   

While processing a batch on a collection, dbCheck holds the database lock in strong shared mode, which unnecessarily interferes with writes to other collections.



 Comments   
Comment by Githook User [ 17/Dec/21 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-61748 dbCheck: improve database-level locking and handling of special collections

  • Acquire database lock in MODE_IS between batches
  • Acquire database lock in MODE_IS when listing collections
  • Remove MODE_IX lock acquisition of the local db
  • Downgrade severity to warning for collections that are not consistent by design
  • Downgrade severity to info for collections that are dropped partway through

(cherry picked from commit ad3ba4b6ae53962de7b943bac4c4c50bd5d85002)
Branch: v4.4
https://github.com/mongodb/mongo/commit/dac8910e21e7e8835087167a6135e10257a7645b

Comment by Githook User [ 15/Dec/21 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-61748 dbCheck: improve database-level locking and handling of special collections

  • Acquire database lock in MODE_IS between batches
  • Acquire database lock in MODE_IS when listing collections
  • Remove MODE_IX lock acquisition of the local db
  • Downgrade severity to warning for collections that are not consistent by design
  • Downgrade severity to info for collections that are dropped partway through

(cherry picked from commit ad3ba4b6ae53962de7b943bac4c4c50bd5d85002)
Branch: v5.0
https://github.com/mongodb/mongo/commit/6f544fdc28219537626cbe5c3a078aa5e389a1b9

Comment by Githook User [ 08/Dec/21 ]

Author:

{'name': 'Josef Ahmad', 'email': 'josef.ahmad@mongodb.com', 'username': 'josefahmad'}

Message: SERVER-61748 dbCheck: improve database-level locking and handling of special collections

  • Acquire database lock in MODE_IS between batches
  • Acquire database lock in MODE_IS when listing collections
  • Remove MODE_IX lock acquisition of the local db
  • Downgrade severity to warning for collections that are not consistent by design
  • Downgrade severity to info for collections that are dropped partway through
    Branch: master
    https://github.com/mongodb/mongo/commit/ad3ba4b6ae53962de7b943bac4c4c50bd5d85002
Comment by Josef Ahmad [ 29/Nov/21 ]

daniel.gottlieb we're in agreement, we were already discussing internally (cc geert.bosch) how to avoid the collection S lock during batches by reading from a timestamp. I've filed SERVER-61754 to track this follow-up.

Comment by Daniel Gottlieb (Inactive) [ 27/Nov/21 ]

josef.ahmad, do you intend to still take a collection X lock? Given a single dbcheck on a collection will turn into a bunch of smaller reads/oplog entries, it's probably worth avoiding that acquisition as well. I think to do that correctly, we'd have to rely on reading at a timestamp, but it's still a bit of a dance to make that work. It may help to allow ourselves to let secondaries "fail" if they don't have history (unlikely).

But if we're trying to expedite this work, I think the ticket as you have it is fine and we can file a different ticket to remove any user-visible side-effects.

Generated at Thu Feb 08 05:53:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.