[SERVER-30893] Default build fails for aarch64 because of missing CCFLAGS option related to CRC Created: 30/Aug/17  Updated: 10/Feb/21  Resolved: 13/Nov/18

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Praveen Arkeri Assignee: Andrew Morrow (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2017-08-30-22-18-44-891.png     PNG File image-2017-08-30-22-18-51-499.png    
Issue Links:
Related
Operating System: ALL
Sprint: Dev Tools 2018-11-19
Participants:

 Description   

On aarch64 platform default build command fails with following error message:

/tmp/ccVvoQKQ.s: Assembler messages:
/tmp/ccVvoQKQ.s:34: Error: selected processor does not support `crc32cb w2,w2,w3'
/tmp/ccVvoQKQ.s:56: Error: selected processor does not support `crc32cx w2,w2,x0'
/tmp/ccVvoQKQ.s:78: Error: selected processor does not support `crc32cb w2,w2,w0'
.
.
scons: *** [build/opt/third_party/wiredtiger/src/checksum/arm64/crc32-arm64.o] Error 1
scons: building terminated because of errors.
build/opt/third_party/wiredtiger/src/checksum/arm64/crc32-arm64.o failed: Error 1

Adding CCFLAGS="-march=armv8-a+crc" to build command successfully builds mongodb binaries for aarch64.



 Comments   
Comment by Billy Donahue [ 05/Jan/21 ]

Maybe a runtime check for the CPU's hw crc instruction?

Compile both implementations of the function and dynamically dispatch to the fastest supported implementation.

Like zlib does?

https://source.chromium.org/chromium/chromium/src/+/master:third_party/zlib/crc32.c;l=293

Or maybe just call our vendored zlib function instead of reinventing it.

Comment by Andrew Morrow (Inactive) [ 05/Jan/21 ]

billy.donahue and daniel.moody - The check needs to be implemented purely at the compile level: interrogating the local machine's CPU isn't appropriate because you may be cross compiling or running on one ARM system but targeting another. If we want to fix this (and I agree it is a nuisance, I've been bitten by it too and ought to know better) I think the right answer is to give --use-hardware-crc32 an auto mode, and then write configure checks for all platforms. The downside to this is that if you do a build from source and fail to name the necessary targeting flags, you will get a degraded performance build since it will autoamatically disable the hardware crc32 support. I don't really like that, as it feels like a potentially huge source of confusion.

Comment by Daniel Moody [ 21/Dec/20 ]

I think it wouldn't be too bad as a configure check, I think having full support of any given platform would be difficult, but I think we could easily create a configure check that covers common and known systems, for example like Billy mentioned. Manual configuration could override the configure check and cover non common systems as well.

Comment by Billy Donahue [ 09/Jan/20 ]

Ran into this problem while trying an unrelated build experiment on aarch64. It's annoying and I wish the hardware detection was done for me, if it's that important.

 
Test for crc32 support might be easy?
Here's an aarch64 spawnhost:

ubuntu@ip-10-122-2-118 $ (grep -q "Features.*crc32" /proc/cpuinfo) && echo "CPU has crc32"
CPU has crc32

Comment by Andrew Morrow (Inactive) [ 13/Nov/18 ]

Arkeri - I'm closing this ticket for now as won't fix, because I don't believe the complexity of implementing the configure time hardware capability check is warranted, but we can re-open it if you have additional thoughts.

Comment by Andrew Morrow (Inactive) [ 07/Nov/18 ]

Arkeri - This is a little tricky. I'm not sure what the best way to proceed here is.

We have a build flag flag --use-hardware-crc32 (note that on v4.0 and older branches the flag is called --use-s390x-crc32, but it still affects arm builds) which controls whether we attempt to use these instructions. So the following two builds will both work for you on arm systems:

scons ... --use-hardware-crc32=off
scons ... --use-hardware-crc32=on CCFLAGS=-march=armv8-a+crc
scons ... CCFLAGS=-march=armv8-a+crc

Where the latter two invocations are the same, since --use-hardware-crc32 defaults to on.

Now, to your point, none of these are "default" builds. But it isn't exactly easy to make that work in a palatable way. The following are some options:

  • We could switch the default for --use-hardware-crc32 to off, and then default builds would work. But this doesn't seem like a good idea, since most users have hardware capable of providing support and the gains are meaningful.
  • We could add an auto} mode for --use-hardware-crc32 and then probe for instruction support in the configure phase and make a choice. This is the best way forward, but is complex, and I'm not sure it is worth the effort.
  • We could do nothing, meaning that default builds for arm wouldn't work on arm, and you would need to either add --use-hardware-crc32=off or the CCFLAGS change to your command line.

Right now, I'm inclined not to make any changes, but if you have thoughts among the above options please let me know.

Comment by Gregory McKeon (Inactive) [ 23/Oct/18 ]

acm I believe this has gone away now? Did WT-3197 fix this?

Generated at Thu Feb 08 04:25:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.