[SERVER-77909] Explore adding $collStats FTDC data for system.preimages Created: 08/Jun/23  Updated: 29/Oct/23  Resolved: 02/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Jordi Olivares Provencio Assignee: Haley Connelly
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by COMPASS-7076 Investigate changes in SERVER-77909: ... Closed
Documented
is documented by DOCS-16295 Investigate changes in SERVER-77909: ... Closed
Duplicate
duplicates SERVER-70599 serverStatus metrics for ChangeStream... Closed
Related
related to SERVER-79492 Consider incrementing PreImagesCollec... Closed
is related to SERVER-79605 Explore adding 'expireAfterSeconds' t... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution EMEA Team 2023-07-10, Execution EMEA Team 2023-07-24, Execution EMEA Team 2023-08-07
Participants:

 Description   

As part of the technical discussion for the truncate refactor of preimages removal, we discovered that currently there's potentially no good way of detecting storage issues with preimages.

To be symmettrical with the oplog, we should explore adding $collStats for preimages in single tenant environments. That way we'll be able to more easily diagnose issues with the collection.



 Comments   
Comment by Haley Connelly [ 02/Aug/23 ]

I filed SERVER-79605  where we can consider adding 'expireAfterSeconds'. Given the expiry on single tenants is either by the oldest oplog timestamp or expireAfterSeconds, the field would only show up if the expireAfterSeconds were set.

Comment by Haley Connelly [ 02/Aug/23 ]

The 'changeStreamPreImages' serverStatus section now takes the following shape on a single tenant environment.

changeStreamPreImages: {
 
       purgingJob: {              
               *// Pre-existing section,  cumulative across tenants in multi-tenant env*                 
                totalPass: <int>,
                docsDeleted: <int>,
                bytesDeleted: <int>,
                scannedCollections: <int>,
                scannedInternalCollections: <int>,
                maxStartWallTimeMillis: <int>,
                timeElapsedMillis: <int>,
        },
 
       // New, exclusive to single-tenant envs, as there is only one pre-images collection
 
       numDocs: <int>,   // Same as $collStats.count
       totalBytes: <int>,   // Same as $collStats.size
 
       // Only included when numDocs is greater than 0 to mimic $collStats behavior.
       avgDocSize: <int> // Same as $collStats.avgObjSize
 
       // The remaining metrics only appear if we can acquire the GlobalLock immediately in MODE_IS. 
       // On very rare occasions, if there is a lock conflict they will be ommitted
       stoargeSize: <int>, // Same as $collStats.storageSize
       freeStorageSize: <int> // Same as $collStats.freeStorageSize         
}

 

Comment by Githook User [ 01/Aug/23 ]

Author:

{'name': 'Haley Connelly', 'email': 'haley.connelly@mongodb.com', 'username': 'haleyConnelly'}

Message: SERVER-77909 Add collStats data to FTDC for changeStreamPreImages
Branch: minh.luu-no_compile_sys-perf
https://github.com/mongodb/mongo/commit/0db8ba9c47d6f56d897d0560f8d8befc086ea3a6

Comment by Githook User [ 31/Jul/23 ]

Author:

{'name': 'Haley Connelly', 'email': 'haley.connelly@mongodb.com', 'username': 'haleyConnelly'}

Message: SERVER-77909 Add collStats data to FTDC for changeStreamPreImages
Branch: master
https://github.com/mongodb/mongo/commit/0db8ba9c47d6f56d897d0560f8d8befc086ea3a6

Comment by Kevin Arhelger [ 18/Jul/23 ]

These diagnostics would ideally be included in FTDC.

Currently, only local.oplog.rs has coll stats included in FTDC, so we'd miss out on the most important metric, which is the total number of documents in the collection. We expose this count for other things like server sessions directly in Server Status.

Generated at Thu Feb 08 06:36:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.