[SERVER-1363] Parallel build error on windows debug build: C:\src\mongo\mongod.pdb : fatal error LNK1212: error opening program database; file is in use Created: 03/Jul/10  Updated: 12/Jul/16  Resolved: 04/Jul/10

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: 1.5.5

Type: Bug Priority: Major - P3
Reporter: Justin Dearing Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Visual Studio 2008/2010


Attachments: Text File build.log    
Operating System: ALL
Participants:

 Description   

My machine has 8 gigs of ram and 4 hyperthreaded codes (8 CPU as far as the OS is concerned) so I build with scons -j8 to peg them all and get the fastest build output. When I need to do something cpu I use -j7 so my system remains responsive.

Full buildlog attached, and I will be doing some further investigation. At the risk of sounding like a broken record I'm new to python so I stumble over stupid stuff a lot.

Theories I have to resolve this problem:

1. Use a phony SideEffect() on everything that invokes lib.exe and linker.exe with mongo.pdb so they don't build in parallel
2. All symbols for all exevcutables seem to be going to one symbol database: env.Append( CPPFLAGS=' /Fd"mongod.pdb" ' )
This should be changes to one pdb per assembly
3.Stop using the PDB construction variable and embed debug symbols directly in the executable.



 Comments   
Comment by Justin Dearing [ 04/Jul/10 ]

Point conceded. I still like my idea of a PDB linker, but this is good enough.

Close this for now and I'll open a new bug if I do find a ways to improve the building/linking that doesn't break parallel builds.

Comment by Eliot Horowitz (Inactive) [ 04/Jul/10 ]

Not sure I agree with that.

In general I think those doing windows development are using visual studio for debugging.

So for those runnings, scons, I think this might be ok...

Comment by Justin Dearing [ 04/Jul/10 ]

Worked, but I request a reversion for reasons discussed above.

Comment by Justin Dearing [ 04/Jul/10 ]

Trying a parallel debug build now. Will let you know in a few if it works.

I don't think we should use /Z7 because its an older debug format with less functionality than /Zi or /ZI. I came across one thread last night where it made a practical difference in that a crash message written to stdout with /Z7 was obscure, but it was quite explicitly about calling convention with /Zi.

Comment by Eliot Horowitz (Inactive) [ 04/Jul/10 ]

See this:

The Visual C++ compiler option that SCons uses by default to generate PDB information is /Z7. This works correctly with parallel (-j) builds because it embeds the debug information in the intermediate object files, as opposed to sharing a single PDB file between multiple object files. This is also the only way to get debug information embedded into a static library. Using the /Zi instead may yield improved link-time performance, although parallel builds will no longer work.

I switched to use Z7 for debug builds.
can you try?

Comment by Justin Dearing [ 03/Jul/10 ]

I agree the redundancy is probably a bad idea. Let me see if I can avoid the need for redundancy.

I don't think it would be that terribly hard to write a PDB linker and generate one PDB from a bunch of PDBs. I know a handful of C/C++ programmers with the IQ and skillset to do that. Just gotta make the task interesting enough for them. And in parallel I've not ruled out the possibility such a tool exists.

The right invocation of cl.exe might link pdbs, after all compiling multiple source files with the sambe pdb output file basically does that.

On another front, if I can find the right Microsoft Evangelist, I might be able to get a feature request into the compiler team at Microsoft to have cl.exe implement some IPC so multiple copies can do shared writes to a pdb file. Thats a little pie in the sky, and over a year away, but Microsoft has been pushing unmanaged programming of the windows 7 UI features. Thats also a useful feature for making use of 8+ core systems, which C++ developers tend to have for workstations.

Comment by Eliot Horowitz (Inactive) [ 03/Jul/10 ]

I don't think the redundancy will end up helping.
And certainly for testing, when you recompile the same binary over and over, this won't help - so don' think its worth it.

Comment by Justin Dearing [ 03/Jul/10 ]

Hope my comment generates some activity on this SO question and maybe I might learn something.
http://stackoverflow.com/questions/528105/how-do-i-merge-multiple-pdb-files

Comment by Justin Dearing [ 03/Jul/10 ]

Ok I did some research, at first glance this should be closed because the /Zi and /ZI switches both write debug info at compile time not link time. So even if each executable had its own pdb file, 2 cpp files for the same executable could not be compiled at the same time.

However, I propose the following behavior based on the behavior I observed from Visual Studio. I'm not sure if this is possible for scons, but let me that that problem upstream. Give me some time before closing this bug please.

1. each executable gets its own PDB file.
2. For windows debug builds obj files get outputted to a special folder, one per executable
2-a. As a corollary to this if a cpp file is used by two executables, it gets compiled twice.
2-b. This is what scons might not be able to do.
3. In some elegant fashion all source files be granted a side effect of their PDB only during windows debug builds
4. Each executable gets a clean target of their pdb file
5. for 32 bit buillds /ZI is used instead of /Zi for its superior debugging capabilities. (Edit and Continue)

This way the build will be parallelized at least to the level of one cl.exe running per target. This is the level of parallelism that I've observed occurring in visual studio. It might not be worth the redundancy, but I might be able to eliminate the redundancy.

Generated at Thu Feb 08 02:56:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.