[CSHARP-256] Debug build used for releases Created: 24/Jun/11  Updated: 02/Apr/15  Resolved: 27/Mar/12

Status: Closed
Project: C# Driver
Component/s: None
Affects Version/s: 1.1
Fix Version/s: 1.4

Type: Bug Priority: Trivial - P5
Reporter: Aristarkh Zagorodnikov Assignee: Sridhar Nanjundeswaran
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive perftest.zip    
Backwards Compatibility: Fully Compatible

 Description   

While doing some performance research on BsonArray creation (some parts of our persistence framework uses arrays heavily so I'm always on a lookout for unnecessary copying, more on that later), I found out that the released 1.1.0.4184 (CSharpDriver-1.1.0.4184.zip from github, didn't check the .msi version) has performance that is lower to the binaries I've built from the same version tag sources. It appears that at least .zip release contains debug versions of binaries (both MongoDB driver and BSON library). I wonder if this is intentional (for example to facilitate better stack traces) or accidental?



 Comments   
Comment by Justin Dearing [ 26/Mar/12 ]

Robert,

If you set the assembly search path in Visual Studio the line numbers show up. Would you like a patch that will do that automatically in the installer? I never automated that, but I'm sure I could figure it out.

Justin

Comment by Robert Stam [ 26/Mar/12 ]

I have not tried setting "Debug Info" to "full" instead of "pdb-only".

Thanks for testing that the Debug build doesn't provide line numbers either when installed in the GAC. Based on that we might as well use the Release build and figure out later if there is a way to get line numbers in stack traces when using a DLL from the GAC.

Comment by Aristarkh Zagorodnikov [ 26/Mar/12 ]

I would also like to note that Debug vs Release builds don't help if you're doing GAC install. I installed the latest 1.3.1 MSI from github, created a new project and added reference to MongoDB.Driver & MongoDB.Bson from the GAC, then called MongoDB.Driver.MongoDatabase.Create((string)null) and still had no line information. Adding MongoDB.*.pdb to the bin/Debug folder of my test application didn't help.

Comment by Aristarkh Zagorodnikov [ 26/Mar/12 ]

Yes, I know this is not the best way, just asking if you tried this or not.

Comment by Aristarkh Zagorodnikov [ 26/Mar/12 ]

I wonder if you changed the "Debug Info" (Project Properties->Build->Advanced...) to "full" instead of "pdb-only" in Release builds?

Comment by Robert Stam [ 26/Mar/12 ]

We are going to postpone changing to Debug release until the next version.

Turns out that if you install the DLLs in the GAC (like the .msi based setup does) then you don't get line numbers in your stack trace even if you have the .pdb files in your executable directory.

We find line number in exceptions reported back to us incredibly useful, so until we can guarantee that we get line numbers in stack traces under all possible deployment scenarios we are sticking to Debug.

Comment by Robert Stam [ 22/Mar/12 ]

Changed where I could find that Debug need to become Release.

Comment by Robert Stam [ 15/Mar/12 ]

The 1.4 release will be created using Release builds.

Comment by Aristarkh Zagorodnikov [ 04/Aug/11 ]

Small performance test that I mentioned in my comment.

Comment by Aristarkh Zagorodnikov [ 04/Aug/11 ]

Unfortunately, none of the answers on the aforementioned Stack Overflow question state that "there may not be a measurable perf difference between the two". The question asks if there is a difference, and the accepted answer stats that it is "Partially true", with Mr. Lippert pointing to a (fairly extensive) list of compiler optimizations (http://blogs.msdn.com/b/ericlippert/archive/2009/06/11/what-does-the-optimize-switch-do.aspx). Some answers make a good point that most of the difference occurs whether you run code under the debugger or not, since JIT is known to suppress specific optimizations when debugger is attached. Many other answers talking about assembly load times, which is not relevant to this topic.

In fact, there is a measurable difference, and while I won't go into lengthy discussion about "why compiler optimization still helps for managed code", I would like to present you a small VS2010 solution (attached), that demonstrates the said difference.
The Driver\Debug contains the BSON library from the 1.1.0.4184 zip archive, while Driver\Release contains the same library built from the source code that corresponds to that version (git rev g29f3acd) in Release configuration.
On my workstation, the resulting code that links with the Debug library can create about 2640 arrays per second, while the one linked with Release build does about 3350, more than 25% improvement.
Yes, I understand that my so-called "performance test" is primitive and synthetic, but you asked for an example =)

Now, a few thoughs about deployment. Personally, I don't care much about deployment itself, since we're building most of the dependencies from source (and already have an MQ patch queue for the mongodb-csharp-driver) , so having something outdated installed in the GAC would do more harm than good to us (and it's easier to bin-deploy to server farms, I can't imagine myself deploying anything into GAC on 10+ machines, while rsync-ing to staging areas and then switching production application with staging one is easy and reliable).
Although, I see almost no point in deploying non-optimized assemblies at all. While in C++ (especially with interprocedural optimization aka WPO), fully optimized binary cannot reliably get it's own stack trace, because of extensive frame pointer omissions and non-conventional parameter passing (so you have to resort to having at least a "partially-optimized" binaries to diagnose problems, along with decoding FPO optimizations and parameter passing conventions when walking the stack), the C# has a much better picture, because the stack is always available (it will omit inlined functions, but that usually doesn't matter, because JIT inlining isn't that aggressive). Yes, there is a small problem with stacks for unmanaged exceptions (NullReferenceException, AccessViolationException, etc.) – the top of the stack may contain wrong line number information due to call sites not being padded by breakpoint no-op "pillows", but I never spend more than a few minutes discovering how "int f = a + b;" can cause an NRE, when "a" and "b" are both local integer variables, so I don't think it's that much of a problem.

Comment by Sridhar Nanjundeswaran [ 03/Aug/11 ]

http://stackoverflow.com/questions/2446027/c-debug-vs-release-performance seems to indicate there may not be a measurable perf difference between the two. Aristarkh can you attach a sample of the code which shows the difference? Thanks. Also we currently have 3 delivery mechanisms for the driver
1) msi
2) zip file
3) nuget packages.
We need to have a solution that allows us to package debug and release in all 3. I propose creating debug and release sub folders for all 3. For msi the question becomes which of the binaries do we install in the GAC by default. Thoughts?

Comment by Justin Dearing [ 23/Jul/11 ]

I think the MSI should ship with the release builds and the PDB. You still get line numbers in stack traces when something goes awry, but you don't get the performance overhead. BTW, I'm willing to be educated that I am wrong about the previous statement, but I know their are other aspects to a debug build besides the PDB database that do negatively affect performance.

Comment by Aristarkh Zagorodnikov [ 27/Jun/11 ]

I believe shipping Release-only binaries would be fine. If one is going to debug the driver code, he'll need sources anyways, so he can build the debug binaries by himself.

Comment by Robert Stam [ 24/Jun/11 ]

It was intentional. But perhaps in the next release we should start shipping Release versions. One possibility would be to ship both. Anyone else have any input?

Generated at Wed Feb 07 21:36:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.