[SERVER-26912] Bulk insert provides little improvement in overall performance with much higher cpu usage Created: 04/Nov/16  Updated: 05/Apr/17  Resolved: 13/Mar/17

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.4.0-rc1
Fix Version/s: None

Type: Question Priority: Minor - P4
Reporter: Nick Judson Assignee: David Hows
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File RC2.bulk-insert.metrics.2016-11-04T18-00-22Z-00000     File bulk-insert.metrics.2016-11-04T17-41-13Z-00000     File single-insert.metrics.2016-11-04T17-33-32Z-00000    
Backwards Compatibility: Fully Compatible
Participants:

 Description   

Windows 2012R2, WT

With an insert-only workload, using the bulk-insert api provides a ~2x improvement in throughput. Using the same test as SERVER-26753, but simply using a bulk-insert of 10 documents (whereas the original test inserted 1), the overall steady-state performance is marginally improved (~3%) but the cpu usage goes from ~55% to 100%.

I realize we are asking more of the DB, but I'm surprised at how little of an improvement there is given the significant increase in CPU usage. Is it possible there is a significant amount of time spent spinning?

Please see the attached diagnostics.



 Comments   
Comment by David Hows [ 13/Mar/17 ]

Hi Nick,

Glad to hear!

I'm marking this one as "Gone away", since the initially reported issue has been resolved in a more recent release.

Thanks for the clarification.

Comment by Nick Judson [ 13/Mar/17 ]

@David Hows - as discussed I ran a series of tests with 3.4.2 and the performance looks excellent with bulk operations. I was able to get the highest throughput yet. Thanks!

Comment by David Hows [ 08/Mar/17 ]

Hi Nick,

Thanks for taking the time to run me through your reproducer.

As discussed, it looks like the issues weren't present in 3.4.2, can you retest locally and confirm.

Thanks!

Comment by David Hows [ 07/Mar/17 ]

Hi nickj, I'm looking to confirm if this issue still exists in recent releases of Mongo.

I was looking here and on SERVER-26753 to get a copy of your workload. I've been unable to get a copy, would you be able to email me a copy? Or can you upload to a provided drop site?

Thanks!

  • David
Comment by Nick Judson [ 04/Nov/16 ]

~Michael Cahill - After re-testing SERVER-26753 with RC2, I see that the fix for that issue isn't merged yet. So be aware that the fix for SERVER-26753 appears to have a big negative impact on this bulk test.

Comment by Nick Judson [ 04/Nov/16 ]

Just tested with the new RC2 and there appears to be a big improvement. Mods please close.

Generated at Thu Feb 08 04:13:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.