[SERVER-50115] arm64/aarch64 support for macOS Created: 05/Aug/20 Updated: 29/Oct/23 Resolved: 02/Apr/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Build |
| Affects Version/s: | 4.4.0 |
| Fix Version/s: | 6.0.0-rc0 |
| Type: | Improvement | Priority: | Minor - P4 |
| Reporter: | Ryan Schmidt | Assignee: | Andrew Morrow (Inactive) |
| Resolution: | Fixed | Votes: | 14 |
| Labels: | platform-request | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||||||||||
| Sprint: | Dev Platform 2022-04-04 | ||||||||||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||||||||||
| Description |
|
Looks like mongodb 4.4.0 supports x86_64 on Linux and macOS, and arm64/aarch64 on Linux but not yet on macOS; I get this error: <built-in>:1:10: fatal error: 'js-confdefs.h' file not found I see lots of js-confdefs.h files, just not in an aarch64/macOS directory: % find . -name js-confdefs.h So we need to add a ./src/third_party/mozjs-60/platform/aarch64/macOS directory with all of its contents. I can try to do it, if you'll let me know how the existing directories were created. |
| Comments |
| Comment by Githook User [ 01/Apr/22 ] | ||
|
Author: {'name': 'Andrew Morrow', 'email': 'acm@mongodb.com', 'username': 'acmorrow'}Message: | ||
| Comment by Anna Henningsen [ 16/Mar/22 ] | ||
|
This is marked as blocked, but all dependency tickets are closed. Is this still blocked at this point? | ||
| Comment by Andrew Morrow (Inactive) [ 18/Jul/21 ] | ||
|
ryandesign - We are still waiting on an ESR release from Mozilla that contains Apple Silicon support, after which we will need to do the work to upgrade to the new release. That work is being tracked in | ||
| Comment by Ryan Schmidt [ 18/Jul/21 ] | ||
|
Was this supposed to have been fixed in 5.0.0? When I try to build 5.0.0 on Apple Silicon without --js-engine=none I get:
With --js-engine=none it builds, as did 4.x. | ||
| Comment by Andrew Morrow (Inactive) [ 19/Aug/20 ] | ||
|
ryandesign - I created the configurations, however, MozJS does not seem to build out of the box, failing somewhere inside WASM support. This suggests that full MacOS ARM support is going to require an upgrade to a MozJS ESR release that supports MacOS ARM. In the meantime, you can build with --js-engine=none to work around this. This will, however, mean that the resulting mongod binary has no $where or other JS based feature support, and the mongo shell won't exist at all. | ||
| Comment by Andrew Morrow (Inactive) [ 19/Aug/20 ] | ||
|
ryandesign - I managed to get it working after grubbing around in some Mozilla bug reports like https://bugzilla.mozilla.org/show_bug.cgi?id=1588340. I added --disable-shared-js to the configure invocation and that seems to have fixed it. What I can't explain is how we managed to generate the existing macOS x86_64 configurations without that. I checked out the original commit where we added ESR 60 and it still didn't work without that flag. The only thing I can figure is that it is something environmental based on running on a newer macOS version. In any event, I have the configs generated and will get them reviewed and committed. Whether the generated JS integration actually functions is another question entirely. | ||
| Comment by Andrew Morrow (Inactive) [ 19/Aug/20 ] | ||
|
I was surprised to hear that the build scripts created a virtualenv - a little digging reveals that it seems to be within one of Mozilla's own build scripts. That is a hassle, as of course is Apple's behavior here: breaking virtualenv is ... a little extreme. So that precludes, at least temporarily, doing a native build. Regarding the attempts to do this on an Intel mac, I don't think we have ever tried to use the MozJS configure process in a cross-compilation setting. It probably was never considered necessary and never attempted. I think your observation that we are saying --host when we ought to be saying --target is likely correct, assuming that the configure script is sane. I don't happen to know what that error you received is about. It is quite obscure. I will try it locally and get back to you. | ||
| Comment by Ryan Schmidt [ 06/Aug/20 ] | ||
|
Thanks for the pointers. Sounds straightforward enough. Here's where I've got to so far. I added these lines to the case statement in gen-config.sh: "platform/aarch64/macOS") I didn't get very far trying to run the script on an ARM Mac. I ran ./gen-config.sh aarch64 macOS but the build process creates a virtualenv and that fails: ERROR: virtualenv is not compatible with this system or executable Apparently creating a virtualenv copies the system's python executable – which was compiled arm64e – into the virtualenv, at which point it's not a blessed system executable anymore; it's just a regular user executable which isn't allowed to be arm64e, so the copied python doesn't work. I switched over to an Intel Mac and got further, but it soon failed with: Traceback (most recent call last): The error occurred while processing the following file or one of the files it includes: /Users/rschmidt/mongodb-src-r4.4.0/src/third_party/mozjs-60/mozilla-release/js/src/build/moz.build The error occurred when validating the result of the execution. The reported error is: The static "js" library is not used in a shared library or a program, but USE_LIBS contains the following shared library names: Maybe you can remove the static "js" library? Have you seen this error before or know what it's talking about? The error is the same if I try to ./gen-config.sh x86_64 macOS without my modifications, even on an older system running macOS 10.13, so it appears just to be broken somehow. P.S: Shouldn't the places where gen-config.sh is using --host in _CONFIG_OPTS be using --target instead? | ||
| Comment by Andrew Morrow (Inactive) [ 05/Aug/20 ] | ||
|
Hi ryandesign - As of right now, macOS on ARM is not a supported platform for MongoDB v4.4, or any other stable MongoDB release. We are in the process of finalizing the platform roadmap for future releases. That proposed roadmap currently includes an investigation into supporting macOS on ARM, and includes backporting any such support to at least v4.4, if possible. Given that, it is interesting to note that the current r4.4.0 sources largely build without modification on the macOS ARM platform, with the exception of SpiderMonkey, for which we do not yet have the necessary platform configuration files, as you have discovered. Producing those files isn't straightforward, and we have not yet investigated whether the vendored version of SpiderMonkey even builds or works on macOS ARM. If not, then adding support for macOS on ARM will require upgrading to a version of SpiderMonkey which does, which is a significant undertaking. There may not even be such a version, yet. Meanwhile, you can build the majority of the tree by using the flag --js-engine=none, which will disable JavaScript entirely. This will of course mean that you don't get a binary for the mongo shell, which is inherently JS based. Finally, I'll note here a few important caveats and other pieces of information that may be helpful while experimenting with building the tree for macOS ARM:
Regarding your specific question about how to generate the configurations for MozJS: you can see this thread from a while back about how to go about it: https://groups.google.com/g/mongodb-dev/c/G-kGjZEEam0/m/VSVB9fYCBAAJ. Obviously some versions have changed, but the general procedure has not. Overall, this all depends on how well the MozJS ESR 60 release happens to do when targeting macOS and ARM simultaneously. If you are interested in trying it out we are of course interested in any results you obtain. Otherwise, this will be one of the first things we pick up if and when we officially start our efforts to target macOS on ARM. Please let us know if there is anything else we can do to help. |