[SERVER-48213] database creation get slower for each databases created Created: 14/May/20  Updated: 01/Jun/20  Resolved: 01/Jun/20

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 4.2.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: emilien puget Assignee: Dmitry Agranat
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File SERVER-48213.png     File databases creation benchmarks - 4.2 databases creation no load.csv     PNG File dhandle lock application thread time waiting.png    
Issue Links:
Duplicate
is duplicated by SERVER-40703 Resource leakage + slowness after cre... Closed
Related
related to SERVER-40703 Resource leakage + slowness after cre... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

I ran this script against a mongo 4.2

https://gist.github.com/emilien-puget/6f9d2f840d4dbe0e613da8c634bf2cc2

 

and got this result

databases creation benchmarks - 4.2 databases creation no load.csv

Participants:

 Description   

Creating databases with indexes takes more time the more databases have been created

 



 Comments   
Comment by emilien puget [ 01/Jun/20 ]

Ok, let's do that

Comment by Dmitry Agranat [ 31/May/20 ]

Thanks emilien@sendinblue.com, we can still see the presence of SERVER-40703 (where the cause of the growing slowness is "dhandle lock application thread time waiting" which is related to the number of dhandles over time):

I suggest closing this as a duplicate of SERVER-40703 and watching SERVER-40703 for progress.

Thanks,
Dima

Comment by emilien puget [ 27/May/20 ]

Sorry, that was another one that crashed because of bad configuration, i uploaded the correct one

Comment by Dmitry Agranat [ 27/May/20 ]

Thanks emilien@sendinblue.com, I see that after creating about 900 tables, the test abruptly ends at 2020-05-25T08:36:46.898Z UTC after which no data is available. Was this intentional?

Comment by emilien puget [ 26/May/20 ]

it's uploaded

Comment by Dmitry Agranat [ 25/May/20 ]

Yes please upload the diagnostic.data when running on 4.4.-rc6 emilien@sendinblue.com

Comment by emilien puget [ 25/May/20 ]

@dmitry.agranat I tried with 4.4-rc6 and haven't got better result, do you want the diagnostic files ?

Comment by Dmitry Agranat [ 21/May/20 ]

Thanks emilien@sendinblue.com, the provided data shows exactly what you are reporting here.

This problem can be broke down into two sub-issues:

  • SERVER-40703 (where the cause of the growing slowness is "dhandle lock application thread time waiting" which is related to the number of dhandles over time)
  • Before MongoDB 4.4, we used MODE_X for creating collections and indexes (we use MODE_IX starting from MongoDB 4.4)

Though SERVER-40703 is still not fixed, I wonder if you could try this with the latest release candidate of MongoDB 4.4 to see if this helps to improve this workload

Thanks,
Dima

Comment by emilien puget [ 19/May/20 ]

Hello @dmitry.agranat , i uploaded the file as requested

Comment by Dmitry Agranat [ 17/May/20 ]

Hi emilien@sendinblue.com,

Would you please archive (tar or zip) the mongod.log files and the $dbpath/diagnostic.data directory (the contents are described here) covering the time of this test and upload them to this support uploader location?

Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Thanks,
Dima

Generated at Thu Feb 08 05:16:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.