[SERVER-51119] Out-of-source builds fail, "mongo/src/mongo/db/sorter/sorter_gen.h: No such file or directory" Created: 23/Sep/20  Updated: 22/Oct/20  Resolved: 22/Oct/20

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: Daniel Moody
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File Workspace.prf    
Issue Links:
Duplicate
is duplicated by SERVER-34484 IDL targets don't work right on exter... Closed
Gantt Dependency
has to be done after SERVER-34484 IDL targets don't work right on exter... Closed
Operating System: ALL
Sprint: Dev Platform 2020-11-02
Participants:

 Description   

I checked out master on Linux and tried to build using a distinct output directory.

Source: "/home/emptysquare/Vermino.us Dropbox/Verminous Davis/mongo"
Build: "/home/emptysquare/build"

> python buildscripts/scons.py --cache CCFLAGS='-gsplit-dwarf' ICECC=icecc CCACHE=ccache --variables-files=etc/scons/mongodbtoolchain_v3_gcc.vars --link-model=dynamic --allocator=system VARIANT_DIR=ninja --ssl --dbg --enable-free-mon=off --install-mode=hygienic --ninja build.ninja --build-dir=/home/emptysquare/build
 
> ninja -f build.ninja -j 400

The build fails with:

FAILED: /home/emptysquare/build/ninja/mongo/client/dbclient_cursor.dyn.o
PATH=/opt/mongodbtoolchain/v3/bin:/usr/local/bin:/opt/bin:/bin:/usr/bin CCACHE_NOCPP2=1 ICECC_VERSION=/home/emptysquare/build/scons/icecc/opt_mongodbtoolcha
in_v3_bin_gcc_opt_mongodbtoolchain_v3_bin_g++.tar.gz CCACHE_PREFIX=/usr/bin/icecc ccache /opt/mongodbtoolchain/v3/bin/g++ @/home/emptysquare/build/ninja/mon
go/client/dbclient_cursor.dyn.o.rsp
In file included from src/mongo/db/storage/storage_engine.h:40,
                 from src/mongo/db/service_context.h:40,
                 from src/mongo/db/client.h:43,
                 from src/mongo/db/operation_context.h:36,
                 from src/mongo/transport/transport_layer.h:37,
                 from src/mongo/client/mongo_uri.h:43,
                 from src/mongo/client/authenticate.h:40,
                 from src/mongo/client/dbclient_base.h:36,
                 from src/mongo/client/connpool.h:35,
                 from src/mongo/client/dbclient_cursor.cpp:42:
/home/emptysquare/build/ninja/mongo/db/resumable_index_builds_gen.h:25:10: fatal error: mongo/src/mongo/db/sorter/sorter_gen.h: No such file or directory
 #include "mongo/src/mongo/db/sorter/sorter_gen.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It seems that /home/emptysquare/build/ninja/mongo/db/resumable_index_builds_gen.h isn't generated correctly. It contains:

#include "mongo/base/data_range.h"
#include "mongo/base/string_data.h"
#include "mongo/bson/bsonobj.h"
#include "mongo/bson/bsonobjbuilder.h"
#include "mongo/bson/simple_bsonobj_comparator.h"
#include "mongo/db/namespace_string.h"
#include "mongo/idl/basic_types.h"
#include "mongo/idl/idl_parser.h"
#include "mongo/rpc/op_msg.h"
#include "mongo/src/mongo/db/sorter/sorter_gen.h"
#include "mongo/util/uuid.h"

I think there's an incompatibility between IDL files that import IDL files and the --build-dir option.



 Comments   
Comment by Daniel Moody [ 21/Oct/20 ]

I tested the commands in the description on ubuntu 18 workstation, and seems to work fine with changes from SERVER-34484. Asked Jesse if this was good enough and he gave me the go ahead to close this issue after those changes merge.

Comment by Ryan Egesdahl (Inactive) [ 08/Oct/20 ]

At first blush, this seems to be completely due to SERVER-34484, as mark.benvenuto suggested. Or, at least, I am not seeing any other causes leaping out at me that haven't already been fixed in some other manner. I think we need to work that issue first, and then we can come back and try this again to see if any other problems pop up with out-of-source builds.

Comment by Mark Benvenuto [ 02/Oct/20 ]

See SERVER-34484 for a related issue about how IDL tries to compute relative paths in the generated files.

Comment by Ryan Egesdahl (Inactive) [ 24/Sep/20 ]

jesse I've attached a sample configuration to this ticket. It depends on the remote host being keyed (the key must be in your keychain or have no password), and you will need to go through the configuration and substitute all instances of $LOCAL_USER, $REMOTE_USER, and $REMOTE_HOST as appropriate. You can also set up your Mac locally to run Unison as a service if you like, but I am afraid I no longer have a Mac that I run Unison on and therefore have no example of a launch daemon configuration handy for you. It's not too hard to do, though, and the one I made a few years ago was essentially copied from a Google search. I hope this helps you work around the problem for now.

Comment by A. Jesse Jiryu Davis [ 24/Sep/20 ]

Thanks, I would appreciate an example Unison config.

Comment by Ryan Egesdahl (Inactive) [ 23/Sep/20 ]

We will definitely look into this, but I may have a solution for you that you can try. Dropbox isn't actually a very good sync platform for this purpose and is likely to cause other issues after this one is resolved. You might want to see if you can get Unison up and running. Make sure both sides of the sync have the same OCAML version (what Unison is written in) and the same Unison version before you start. I have a config I can share with you that can get you started.

Comment by A. Jesse Jiryu Davis [ 23/Sep/20 ]

This problem arose because I'm experimenting with a new way to edit code on Mac and build on Linux. I want to share a Dropbox folder between my Mac and Linux machines. I want to sync only the code, not the build artifacts, so I tried an out-of-source build and hit this error.

Generated at Thu Feb 08 05:24:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.