[SERVER-43910] include Client/OpCtx information in LockManager::dump() output Created: 09/Oct/19  Updated: 29/Oct/23  Resolved: 16/Oct/19

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.3.1, 4.2.3, 4.0.15

Type: Improvement Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
causes SERVER-45831 [4.0] Performance regression processi... Closed
Related
related to SERVER-45012 Add transaction information to lockInfo Closed
is related to SERVER-16203 Add command to dump lock manager state Closed
is related to SERVER-39860 Separate reporting of RSTL and PBWM l... Closed
is related to SERVER-37406 A stashed Locker should keep informat... Closed
is related to SERVER-39902 lockInfo should use UUIDCatalog to ma... Closed
is related to SERVER-40915 Show view namespaces in the LockManag... Closed
is related to SERVER-40780 LockManager dump should show the name... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.2, v4.0
Sprint: Execution Team 2019-10-21
Participants:
Linked BF Score: 0

 Description   

As a diagnostic tool, the LockManager::dump() function, when invoked through an attached debugger, logs a list of acquired and enqueued locks. The lockInfo command also performs a similar function through the shell/client interface, but with additional information on the Client/OperationContext objects associated with each granted/pending lock request. Since the LockManager::dump() function is used more widely as part of the shutdown procedure for stalled test runs in our CI system, it would improve diagnostics to also include the Client/OpCtx information in the logs() when dumping the lock manager's internal state through the debugger.

Sample output:

Dumping LockManager @ 0x10e62e590
Lock @ 0x7F891DD23920: {2305843009213693953: ParallelBatchWriterMode, 1}
GRANTED:
	LockRequest 223 @ 0x7F892E87CE00: Mode = X; Thread = 0x70000b7e0000 | 0x70000b7e0000; ConvertMode = NONE; EnqueueAtFront = 1; CompatibleFirst = 1; DebugInfo = ; ClientInfo = { desc: "rsSync-0", opid: 220 }
PENDING:
	LockRequest 240 @ 0x7F88FE074000: Mode = IS; Thread = 0x70000c5b1000 | 0x70000c5b1000; ConvertMode = NONE; EnqueueAtFront = 0; CompatibleFirst = 0; DebugInfo = ; ClientInfo = { desc: "conn12", connectionId: 12, client: "127.0.0.1:56489", opid: 236 }
-----------------------------------------------------------

Sample output with command information:

Dumping LockManager @ 0x111458590
Lock @ 0x7F8654D230E0: {2305843009213693953: ParallelBatchWriterMode, 1}
GRANTED:
	LockRequest 248 @ 0x7F865580DC00: Mode = X; Thread = 0x70000eca3000 | 0x70000eca3000; ConvertMode = NONE; EnqueueAtFront = 1; CompatibleFirst = 1; DebugInfo = ; ClientInfo = { desc: "rsSync-0", opid: 245 }
PENDING:
	LockRequest 265 @ 0x7F8656801E00: Mode = IS; Thread = 0x70000fa74000 | 0x70000fa74000; ConvertMode = NONE; EnqueueAtFront = 0; CompatibleFirst = 0; DebugInfo = { listDatabases: 1.0, lsid: { id: UUID("78521e0f-7d7e-40ba-a7fc-44d2d2813ebc") }, $clusterTime: { clusterTime: Timestamp(1570622648, 2), signature: { hash: BinData(0, 0000000000000000000000000000000000000000), keyId: 0 } }, $readPreference: { mode: "secondaryPreferred" }, $db: "admin" }; ClientInfo = { desc: "conn12", connectionId: 12, client: "127.0.0.1:56358", opid: 261 }
-----------------------------------------------------------



 Comments   
Comment by Benety Goh [ 29/Jan/20 ]

Due to SERVER-45831 (affecting the 4.0 branch only), we had to remove the redacted command info from the DebugInfo field. The ClientInfo field containing the connection name and opid should still be available in the LockManager output.

Comment by Githook User [ 29/Jan/20 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: Revert "SERVER-43910 BasicCommand::Invocation sets LockManager DebugInfo"

This reverts commit 41981ad8d67b3fc95da21506751cc5d284e724e1.
Branch: v4.0
https://github.com/mongodb/mongo/commit/f718c0e820777c02af3577905876d63936a25d55

Comment by Benety Goh [ 22/Jan/20 ]

Linking to SERVER-37406 because of the 4.0 backport dependency.

Comment by Githook User [ 15/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 LockManager::dump() includes client info

(cherry picked from commit 1245770e0ac12d09bceec097fcf66865b040a463)
Branch: v4.0
https://github.com/mongodb/mongo/commit/ce0a2242ca1d939cc2102857d5def9e068ebebe8

Comment by Githook User [ 15/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 BasicCommand::Invocation sets LockManager DebugInfo

(cherry picked from commit 34719559a9ec4bd494a59d269d227d06b086affb)
Branch: v4.0
https://github.com/mongodb/mongo/commit/41981ad8d67b3fc95da21506751cc5d284e724e1

Comment by Githook User [ 15/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 lockInfo command obtains lock to client info mapping from LockManager

(cherry picked from commit cf4c944977a348494d81eeaf7eddb96ef0457876)
Branch: v4.0
https://github.com/mongodb/mongo/commit/e4fd2a68f21e10573dfb1b2ed509b3ca102a466e

Comment by Githook User [ 15/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 Remove usage of BSONObjBuilder append unsigned method from lockInfo command

(partially cherry picked from commit 990b2ef30d7b8fa4db6af6d79da80e3664df9c21)
Branch: v4.0
https://github.com/mongodb/mongo/commit/3f80c90ce6b96ffba44f97f93dfbb7aa7fc85608

Comment by Githook User [ 15/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 lockInfo command output includes Locker::getDebugInfo()

(cherry picked from commit 078f31ba54768fa5945c122257e740f1196defe8)
Branch: v4.0
https://github.com/mongodb/mongo/commit/b4aec2e705d1253c6ec7ee9e91ff091bf742df81

Comment by Githook User [ 09/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 LockManager::dump() includes client info

(cherry picked from commit 1245770e0ac12d09bceec097fcf66865b040a463)
Branch: v4.2
https://github.com/mongodb/mongo/commit/836ed40fb80cb277e254c735f8a5a91c4311ebfb

Comment by Githook User [ 08/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 BasicCommand::Invocation sets LockManager DebugInfo

(cherry picked from commit 34719559a9ec4bd494a59d269d227d06b086affb)
Branch: v4.2
https://github.com/mongodb/mongo/commit/e2afa9a595a04e4828097c290c2c97beb5a22620

Comment by Githook User [ 08/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 lockInfo command obtains lock to client info mapping from LockManager

(cherry picked from commit cf4c944977a348494d81eeaf7eddb96ef0457876)
Branch: v4.2
https://github.com/mongodb/mongo/commit/4c9f3c7b743fc02ee45151eca3e2707d3db32762

Comment by Githook User [ 08/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 Remove usage of BSONObjBuilder append unsigned method from lockInfo command

(partially cherry picked from commit 990b2ef30d7b8fa4db6af6d79da80e3664df9c21)
Branch: v4.2
https://github.com/mongodb/mongo/commit/ab9e42787dea6c166f3987d541c0ddb7b78e25fa

Comment by Githook User [ 08/Jan/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-43910 lockInfo command output includes Locker::getDebugInfo()

(cherry picked from commit 078f31ba54768fa5945c122257e740f1196defe8)
Branch: v4.2
https://github.com/mongodb/mongo/commit/d95377677e832cf62f40f29d74b0b2fffe65a5a3

Comment by Benety Goh [ 10/Dec/19 ]

bartle, the changes in this ticket will surface the command invocation information to the lock manager output. We still have to verify that the "lsid" and "txnNumber" fields mentioned in SERVER-45012 will be provided by the command invocation subsystem in 4.2 and 4.0.

Comment by Benety Goh [ 10/Dec/19 ]

This was originally done to support some 4.4 development work. The changes themselves don't seem that controversial. I've put in the request to start the back port process.

Comment by David Bartley [ 07/Dec/19 ]

Oh, whoops, meant SERVER-45012

Comment by David Bartley [ 07/Dec/19 ]

Would it be possible to backport this to 4.2 and 4.0? We're trying to debug a deadlock (see SERVER-43910), and I think this would surface txnNumber and lsid in lockInfo's output? Right now we're just seeing granted entries of the form:

      {
        "compatibleFirst": false,
        "convertMode": "NONE",
        "enqueueAtFront": false,
        "mode": "IX"
      },

Comment by Githook User [ 11/Oct/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-43910 LockManager::dump() includes client info
Branch: master
https://github.com/mongodb/mongo/commit/1245770e0ac12d09bceec097fcf66865b040a463

Comment by Githook User [ 11/Oct/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-43910 BasicCommand::Invocation sets LockManager DebugInfo
Branch: master
https://github.com/mongodb/mongo/commit/34719559a9ec4bd494a59d269d227d06b086affb

Comment by Githook User [ 11/Oct/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-43910 lockInfo command obtains lock to client info mapping from LockManager
Branch: master
https://github.com/mongodb/mongo/commit/cf4c944977a348494d81eeaf7eddb96ef0457876

Comment by Githook User [ 11/Oct/19 ]

Author:

{'username': 'benety', 'email': 'benety@mongodb.com', 'name': 'Benety Goh'}

Message: SERVER-43910 lockInfo command output includes Locker::getDebugInfo()
Branch: master
https://github.com/mongodb/mongo/commit/078f31ba54768fa5945c122257e740f1196defe8

Comment by Benety Goh [ 09/Oct/19 ]

The lockInfo command was added in SERVER-16203.

Generated at Thu Feb 08 05:04:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.