[DOCS-12332] Docs for SERVER-38240: Extend OpDebug object to support storage statistics Created: 07/Jan/19  Updated: 13/Nov/23  Resolved: 09/Apr/19

Status: Closed
Project: Documentation
Component/s: manual, Server
Affects Version/s: None
Fix Version/s: 4.1.7, 4.0.9, Server_Docs_20231030, Server_Docs_20231106, Server_Docs_20231105, Server_Docs_20231113

Type: Task Priority: Major - P3
Reporter: Kay Kim (Inactive) Assignee: Kay Kim (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-38240 Extend OpDebug object to support stor... Closed
is documented by SERVER-38243 Test session storage statistics are p... Closed
Related
Participants:
Days since reply: 4 years, 43 weeks, 1 day ago
Epic Link: DOCS: 4.2 Server/Tools

 Description   

Description

Description:

With this change, the slow operation's log messages could additionally have the storage statistics information like shown below :

2018-07-05T14:13:21.744+1000 I COMMAND  [conn1] command test.coll appName: "MongoDB Shell" command: insert { insert: "coll", ordered: true, $db: "test" } ninserted:1 keysInserted:1 numYields:0 reslen:45 
locks:{ Global: { acquireCount: { r: 3, w: 3 } }, Database: { acquireCount: { w: 2, W: 1 } }, Collection: { acquireCount: { w: 2 }}}, storage: { data: { bytesRead: 1, bytesWritten: 123, timeReadingMicros: 123, timeWritingMicros: 231 }, timeWaitingMicros: { cache: 2, schemaLock: 1, handleLock: 3}} protocol:op_msg 12ms

The system.profile collection can also contain the additional storage statistics information like this:

{"op":"query","ns":"test.foo.abcd","command":{"find":"foo.abcd","filter":{},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1537768648,"i":1}},"signature":{"hash":{"$binary":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","$type":"00"},"keyId":{"$numberLong":"0"}}},"$db":"test"},"keysExamined":0,"docsExamined":6,"cursorExhausted":true,"numYield":0,"locks":{"Global":{"acquireCount":{"r":{"$numberLong":"2"}}},"Database":{"acquireCount":{"r":{"$numberLong":"1"}}},"Collection":{"acquireCount":{"r":{"$numberLong":"1"}}}},"storage":{"data":{{"bytesRead":{"$numberLong":"1"}}},{"bytesWritten":{"$numberLong":"123"}},{"timeReadingMicros":{"$numberLong":"123"}}},{"timeWritingMicros":{"$numberLong":"123"}},"timeWaitingMicros":{{"cache":{"$numberLong":"2"}},{"schemaLock":{"$numberLong":"123"}},{"handleLock":{"$numberLong":"3"}},"nreturned":6,"responseLength":3965,"protocol":"op_msg","millis":0,"planSummary":"COLLSCAN","execStats":{"stage":"COLLSCAN","nReturned":6,"executionTimeMillisEstimate":0,"works":8,"advanced":6,"needTime":1,"needYield":0,"saveState":0,"restoreState":0,"isEOF":1,"invalidates":0,"direction":"forward","docsExamined":6},"ts":{"$date":"2018-09-24T05:57:31.071Z"},"client":"127.0.0.1","appName":"MongoDB Shell","allUsers":[],"user":""}

"storage" : {
      "data" : {
         "bytesRead" : NumberLong(2097229),
         "timeReadingMicros" : NumberLong(8116)
         "bytesWritten" : NumberLong(4194458),
         "timeWritingMicros" : NumberLong(203)
        }
       "timeWaitingMicros" : {
         "cache" : NumberLong(558),
         "schemaLock": NumberLong(442),
         "handleLock" : NumberLong(342)
        }
    },

The description for the new fields:

system.profile.storage.data.bytesRead : Number of bytes read by the operation from the disk to the cache.
system.profile.storage.data.timeReadingMicros : Time in microseconds that the operation had to spend to read from the disk.
system.profile.storage.data.bytesWritten : Number of bytes written by the operation from the cache to the disk.
system.profile.storage.data.timeWritingMicros : Time in microseconds that the operation had to spend to write to the disk.
system.profile.storage.timeWaitingMicros.cache : Time in microseconds that the operation had to wait for space in the cache.
system.profile.storage.timeWaitingMicros.schemaLock : Time in microseconds that the operation had to wait to acquire the schemaLock.
system.profile.storage.timeWaitingMicros.handleLock : Time in microseconds that the operation had to wait to acquire the handleLock.

Engineering Ticket Description:

Add boost::optional<BSONObj> OperationStorageStats and provide setter method to populate it with storage statistics.
Extend OpDebug::append() and OpDebug::report() to use OperationStorageStats for report generation on storage statistics.

Scope of changes

Impact to Other Docs

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 16/Apr/19 ]

Author:

{'name': 'Kay Kim', 'email': 'kay.kim@mongodb.com'}

Message: DOCS-12332: 4.2 storage metrics for slow ops
Branch: v4.0
https://github.com/mongodb/docs/commit/9c39edd9fa9dac5bfcdef98ba4aa8107eeadcadb

Comment by Githook User [ 09/Apr/19 ]

Author:

{'email': 'kay.kim@10gen.com', 'name': 'Kay Kim', 'username': 'kay-kim'}

Message: DOCS-12332: tweak for 4.0.9 backport
Branch: master
https://github.com/mongodb/docs/commit/37d430567fd92d3a87813fc9aafa1420a911e111

Comment by Githook User [ 09/Apr/19 ]

Author:

{'email': 'kay.kim@mongodb.com', 'name': 'Kay Kim'}

Message: DOCS-12332: 4.2 storage metrics for slow ops
Branch: v4.0.9
https://github.com/mongodb/docs/commit/d68efea0eb50e1d93237bfdd8640e26774b868fd

Comment by Sulabh Mahajan [ 09/Apr/19 ]

Hey kay.kim, the backports for the storage statistics for slow operations is complete now. The documentation here should be updated for 4.0 as well.

Comment by Kay Kim (Inactive) [ 21/Feb/19 ]

Closing for now since the backport ticket itself is blocked.
We have a sync script we run that will update fix versions in our docs ticket. So, we can catch the new version when the backport ticket completes.

Comment by Kay Kim (Inactive) [ 28/Jan/19 ]

jocelyn.del-prado - that's fine.

Comment by Jocelyn del Prado [ 28/Jan/19 ]

Hi kay.kim, This will be backported to 4.0 so this documentation also applies to 4.0 . I've re-opened the ticket but please let me know if you prefer a separate ticket.

Thanks, Jocelyn

Comment by Githook User [ 17/Jan/19 ]

Author:

{'email': 'kay.kim@mongodb.com', 'name': 'Kay Kim'}

Message: DOCS-12332: 4.2 storage metrics for slow ops
Branch: master
https://github.com/mongodb/docs/commit/e88eec14735f792ec87f146b93d9eaa11bb53fc4

Generated at Thu Feb 08 08:04:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.