[CSHARP-1053] Major performance degradation from 1.6.1 to 1.9.2 Created: 28/Aug/14 Updated: 04/Apr/15 Resolved: 04/Apr/15 |
|
| Status: | Closed |
| Project: | C# Driver |
| Component/s: | Performance |
| Affects Version/s: | 1.9, 1.9.1, 1.9.2 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Anthony Sutton | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Description |
|
I have thrown together a performance demonstration for this ticket here: https://github.com/squirrelsama/mongo-csharp-driver-performance-issue The short of it: When my team upgraded the driver from 1.6.1 to 1.9.2, we were hit with serious performance issues and had to roll back. We upgraded specifically because we need the ReadEncoding setting support introduced in 1.8 and will use 1.8 until this issue gets addressed. More details in the github link, including a runnable perf script that takes ~10 minutes. I believe this issue is major because with the 1.6.1 driver, mongo will perform the test in just under 30 seconds; but in 1.9.2, that time doubles to 60 seconds. This likely implies that at minimum, half the time spent trying to interact with mongodb is spent in the driver's code, not waiting on mongodb's network. If you want more information, I'd be more than happy to oblige. |
| Comments |
| Comment by Craig Wilson [ 04/Sep/14 ] |
|
Hi Anthony, Thanks for the report. I'm concerned a little that even 1.8.3 is worse than 1.6. However, 1.9 is definitely going to be slower in certain cases. The server now prefers the use of write commands, and there is a lot of logic that goes into splitting batches correctly and us needing to decide whether to use the old opcodes or the new write commands. Using 1.6 will prevent you from using newer versions of the server when they remove the opcodes (yet to be determined). In addition, if you are using server versions 2.6 in a sharded environment, there is no getting around this overhead at all as mongos does the same thing we are doing. Long and short... we will work on improving performance and I apologize for the drift. Part of it is unavoidable, but we'll see what we can do to amp it back up. Craig |
| Comment by Anthony Sutton [ 04/Sep/14 ] |
|
Attached are some mongo stats for deploying from 1.6.1 to 1.8.3. For now, my plan is to fork 1.6.1 to add the ReadEncoding feature we need, and defer upgrading until these performance issues are resolved. |
| Comment by Anthony Sutton [ 29/Aug/14 ] |
|
FYI, just to clarify: We did observe the initial performance issue that required rollback with the client running on Windows, mongod running on Linux. Only my profiling was done with the client running on Linux/mono |
| Comment by Jose Luis Pedrosa [ 29/Aug/14 ] |
|
Attached the profiling from 1.6. |
| Comment by Jose Luis Pedrosa [ 29/Aug/14 ] |
|
profiling from visual studio running the test of 1.9.2 |
| Comment by Jose Luis Pedrosa [ 29/Aug/14 ] |
|
Hi, I've run the test in windows 7 64 bits, Release version. this are the results: from VS From cmdline Indeed just by browsing the task manager, I can see that in version 1.6 the mongod proc uses more CPU than the load test generator, while on 1.9 it's 50%-50%. The diferences are not as high as the one that Anthony had with mono. Looking at the performance profiling, aparently, Insert operation is the one that has become heavier. BR |
| Comment by Anthony Sutton [ 28/Aug/14 ] |
|
Typo at the end: "not waiting on mongodb OR the network." |