[SERVER-65139] Improve namespace fetching performance in BackupBlock::_initialize() Created: 31/Mar/22  Updated: 29/Oct/23  Resolved: 01/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.0.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Gregory Wlodarek
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File after.svg     File before.svg     File fixed.svg     File repro.js    
Issue Links:
Depends
Problem/Incident
is caused by SERVER-62427 Files mapping to namespaces in a back... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2022-04-04, Execution Team 2022-04-18
Participants:
Linked BF Score: 135

 Description   

As of SERVER-62427, files mapping to namespaces will include the namespace in the backup cursor response.

This function currently has a quadratic runtime.

One approach to improve performance could be to store the catalog entries outside of this class, so we don't have to fetch them for each backup block, and pass them in sorted into this function.

 

Backup cursor performance as of SERVER-62427 (~4 minutes, 10000 collections):

[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:22:13.282+00:00"},"s":"I",  "c":"STORAGE",  "id":24200,   "ctx":"conn1","msg":"Opened backup cursor","attr":{"backupId":{"uuid":{"$uuid":"2d400d44-082e-4da7-92db-0399e3513c28"}},"term":1}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:22:22.686+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd.aggregate","appName":"MongoDB Shell","command":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":5957148721134066174,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":101,"reslen":14873,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":3}},"Global":{"acquireCount":{"r":2}}},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"storage":{},"remote":"127.0.0.1:39852","protocol":"op_msg","durationMillis":9594}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:26:20.941+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd","appName":"MongoDB Shell","command":{"getMore":5957148721134066174,"collection":"$cmd.aggregate","lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"originatingCommand":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"c4cd6a2e-b41c-489f-88d8-5320700b88ab"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648750902,"i":7}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":5957148721134066174,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":19943,"reslen":2857705,"locks":{},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"remote":"127.0.0.1:39852","protocol":"op_msg","durationMillis":238253}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:26:20.978+00:00"},"s":"I",  "c":"STORAGE",  "id":24203,   "ctx":"conn1","msg":"Closed backup cursor","attr":{"backupId":{"uuid":{"$uuid":"2d400d44-082e-4da7-92db-0399e3513c28"}}}}

Backup cursor performance prior to SERVER-62427 (<1 second, 10000 collections):

[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:37.900+00:00"},"s":"I",  "c":"STORAGE",  "id":24200,   "ctx":"conn1","msg":"Opened backup cursor","attr":{"backupId":{"uuid":{"$uuid":"f8d60327-892f-4313-ad25-618f3e256b8c"}},"term":1}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:37.903+00:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn1","msg":"Slow query","attr":{"type":"command","ns":"admin.$cmd.aggregate","appName":"MongoDB Shell","command":{"aggregate":1,"pipeline":[{"$backupCursor":{}}],"cursor":{},"lsid":{"id":{"$uuid":"40ebb2e0-2337-442a-af0e-9284de06bc1f"}},"$clusterTime":{"clusterTime":{"$timestamp":{"t":1648752426,"i":31}},"signature":{"hash":{"$binary":{"base64":"AAAAAAAAAAAAAAAAAAAAAAAAAAA=","subType":"0"}},"keyId":0}},"$readPreference":{"mode":"secondaryPreferred"},"$db":"admin"},"cursorid":3445142455559510098,"keysExamined":0,"docsExamined":0,"numYields":0,"nreturned":101,"reslen":13084,"locks":{"ReplicationStateTransition":{"acquireCount":{"w":3}},"Global":{"acquireCount":{"r":2}}},"readConcern":{"level":"local","provenance":"implicitDefault"},"writeConcern":{"w":"majority","wtimeout":0,"provenance":"implicitDefault"},"storage":{},"remote":"127.0.0.1:41894","protocol":"op_msg","durationMillis":176}}
[js_test:repro] d20020| {"t":{"$date":"2022-03-31T18:47:38.006+00:00"},"s":"I",  "c":"STORAGE",  "id":24203,   "ctx":"conn1","msg":"Closed backup cursor","attr":{"backupId":{"uuid":{"$uuid":"f8d60327-892f-4313-ad25-618f3e256b8c"}}}}



 Comments   
Comment by Githook User [ 04/Apr/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-65139 Take global lock before fetching catalog entries
Branch: master
https://github.com/mongodb/mongo/commit/552f88138a0d5caeeb32f351019801b48254d1cd

Comment by Githook User [ 01/Apr/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-65139 Improve namespace fetching performance in BackupBlock::_initialize()
Branch: master
https://github.com/mongodb/mongo/commit/cce0c09eb1ef0895835368e10d9138c7903ff58d

Comment by Githook User [ 01/Apr/22 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: SERVER-65139 Improve namespace fetching performance in BackupBlock::_initialize()
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/fdf769a5ceb65d381b625a995fe2c7cce1d3d598

Generated at Thu Feb 08 06:01:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.