[SERVER-34717] Performance regression in dropDatabase Created: 27/Apr/18  Updated: 27/Oct/23  Resolved: 06/Apr/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 3.7.5
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Daniel Gottlieb (Inactive)
Resolution: Gone away Votes: 0
Labels: nyc
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Histogram.png     PNG File chart.png    
Issue Links:
Related
is related to SERVER-34713 Progressively declining dropDatabase ... Closed
is related to SERVER-34372 Drop collection command with majority... Closed
Operating System: ALL
Sprint: Execution Team 2020-04-20
Participants:

 Description   

    for (var j = 0; j < 20; j++) {
        for (var i = 0; i < 1000; i++) {
            db["c"+i].insert({})
        }
        t = new Date()
        db.dropDatabase()
        print(new Date() - t)
    }

Time required for dropDatabase has substantially increased in 3.7.5. Like in 3.6.4 it progressively increases on each iteration of dropDatabase (which is the subject of SERVER-34713), although the coefficient may be smaller than 3.6.4. Variability appears to be quite a bit larger.



 Comments   
Comment by Daniel Gottlieb (Inactive) [ 06/Apr/20 ]

Closing as "Gone away".

Comment by Bruce Lucas (Inactive) [ 06/Apr/20 ]

Re-ran on my machine and saw

  • ~1s and steady for both 3.4.10 and 3.4.15
  • ~5s and increasing for 3.6 and 4.0
  • ~1.5s and steady for 4.2, 4.3.5, and master

So I agree it appears that the steadily worsening performance went away in 4.2. There is still possibly a performance difference between 4.2+ and 3.4 that could be investigated, but given the performance in 3.6 and 4.0 it seems a moot point now. So I'm happy to close this as "gone away" if you are.

Comment by Daniel Gottlieb (Inactive) [ 05/Apr/20 ]

Disregard the "histogram" title; I don't know how to google sheets. This is over 1600 runs and I'm not seeing any appreciable upward trend in latency. This is my ~24 hour run against master with EMRC: on

Comment by Daniel Gottlieb (Inactive) [ 03/Apr/20 ]

bruce.lucas can you rerun your experiment against some recent master/4.4-ish build? I'm getting a lot of variability and am unable to reproduce the order of magnitude you have. With 3.4.10 your graph shows ~500ms, while my 3.4.15 is at ~2s with some bumps to ~7-8 seconds. The good news on my run though is that master with EMRC=on (default, includes new two-phase drop) is ~400-500ms. EMRC=off (uses old two-phase drop) is ~3seconds. If my numbers are trending upwards:
a) The variance may be masking it.
b) The improvement from new two-phase drop may negate the need for further improvements.

In the meantime I'll gather data from a longer running experiment.

Comment by Benety Goh [ 27/Apr/18 ]

We may be able to rule SERVER-34372 out as a root cause because we introduced it in 3.7.6.

Generated at Thu Feb 08 04:37:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.