[SERVER-55476] Database profiling should not serialize writes on all collections Created: 24/Mar/21  Updated: 27/Oct/23  Resolved: 06/May/22

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Dmitry Agranat Assignee: Backlog - Storage Execution Team
Resolution: Gone away Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File profiling.png    
Issue Links:
Depends
depends on SERVER-18934 Don't require storage engines to impl... Backlog
Assigned Teams:
Storage Execution
Sprint: Execution Team 2021-11-15
Participants:
Case:

 Description   

Capped inserts on the primary are serialized because capped inserts on secondaries need to be serialized, and we need both primary and secondary to have the same level of write parallelism. Capped inserts on the secondary need to be serialized because we can only use the oplog to determine the ordering. This is a problem for a the system.profile collection which we use for Database profiling, since, in order to achieve the above, we use the metadata lock acquisition. This effectively produces database-level exclusive locking granularity for both reads and writes when the profiler is enabled.



 Comments   
Comment by Louis Williams [ 06/May/22 ]

This was fixed (unintentionally) by SERVER-61277 in version 5.3.0. The profiler no longer introduces a bottleneck by serializing reads and writes on the database.

What used to be true about all local collections is now true for the system.profile collection: tailable cursors may miss writes when operations are inserted out-of-order.

I have filed SERVER-66283 to consider banning tailable cursors on unreplicated capped collections entirely.

Comment by Louis Williams [ 15/Nov/21 ]

It will be very straightforward to stop taking the capped lock for unreplicated collections. As mentioned, this will break tailable cursors, however.

I'm not particularly interested in finding out whether any users depend on being able to tail the profile collection, so I have an idea. There has been some renewed interest inĀ SERVER-18934, which would track oplog visibility above the storage engine. Implemented correctly, we could extend this visibility tracking logic to all capped collections. In this way, we can allow concurrent inserts into the profile collection and continue supporting tailable cursors.

Comment by Louis Williams [ 30/Apr/21 ]

Note that capped deletes are still serialized by a mutex. So while this improvement will allow unreplicated capped collections to avoid taking the X metadata lock for the duration of the insert operation, there will still be a small critical section for operations that are tasked with deleting excess documents.

Comment by Bruce Lucas (Inactive) [ 24/Mar/21 ]

I think we might also need to consider the relative cost of truncating a capped collection vs removing expired entries from a ttl collection.

Comment by Louis Williams [ 24/Mar/21 ]

The system.profile collection is a special case for capped collections because it is not replicated. We already avoid taking the metadata lock when the collection is on the local database. I think we should consider a solution where we extend this to avoid taking the metadata lock on special unreplicated collections like system.profile.

Generated at Thu Feb 08 05:36:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.