[GODRIVER-1727] Allow static compilation when using client-side encryption Created: 24/Aug/20  Updated: 27/Oct/23  Resolved: 23/Mar/23

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: 1.3.7
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Ido Perlmuter Assignee: Preston Vasquez
Resolution: Gone away Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Alpine Linux 3.12
Go 1.14.4


Issue Links:
Depends
depends on MONGOCRYPT-277 libmongocrypt-static pkg-config scrip... Closed
Quarter: FY24Q1

 Description   

When compiling Go programs that use the MongoDB driver with the "cse" tag (to enable client-side encryption), static compilation will fail. To be clear, by static compilation I mean the following go build command is used:

GOOS=linux GOARCH=amd64 go build -tags cse --ldflags '-linkmode external -extldflags "-static"'

This results in the following error:

/usr/local/go/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/lib/gcc/x86_64-alpine-linux-musl/9.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lmongocrypt
collect2: error: ld returned 1 exit status

Static compilation is required to be able to package & deploy such programs as AWS Lambda functions (which are simple zip files).

The issue stems from line 11 at the x/mongo/driver/mongocrypt/mongocrypt.go file:

 // #cgo linux solaris darwin pkg-config: libmongocrypt

This can never statically compile because the installation process of libmongocrypt generates two separate pkg-config files: libmongocrypt.pc, and libmongocrypt-static.pc, when in fact one .pc file can be used for both, and the --static flag to pkg-config can be used to return appropriate flags for static compilation.

To be able to statically compile, I had to replace the aforementioned line with the following:

// #cgo linux CFLAGS: -I/usr/include/mongocrypt -I/usr/include/kms_message -I/usr/include/libbson-1.0 -DBSON_STATIC
// #cgo linux LDFLAGS: /usr/lib64/libmongocrypt-static.a /usr/lib64/libkms_message-static.a -L/usr/lib64 -lbson-static-1.0 /usr/lib/librt.a /usr/lib/libm.a -lssl -lcrypto -ldl -pthread

This is obviously very specific to my environment, but I am wondering if there is a way to allow static compilation without having to fork the driver and change this one line.



 Comments   
Comment by PM Bot [ 23/Mar/23 ]

There hasn't been any recent activity on this ticket, so we're resolving it. Thanks for reaching out! Please feel free to comment on this if you're able to provide more information.

Comment by PM Bot [ 08/Mar/23 ]

Hey ido@ido50.net, We need additional details to investigate the problem. If this is still an issue for you, please provide the requested information.

Comment by Preston Vasquez [ 28/Feb/23 ]

ido@ido50.net From the cgo documentation:

The default pkg-config tool may be changed by setting the PKG_CONFIG environment variable.

That is, one solution to avoid forking the repository would be to build the library pointing to a custom libmongocrypt.pc file. For example,

PKG_CONFIG_PATH="/etc/opts/mongocrypt/pkgconfig" \
    CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build \
    -tags 'cse' \
    -ldflags '-linkmode external -extldflags "-static"'

Then in "/etc/opts/mongocrypt/pkgconfig/libmongocrypt.pc" you can add your custom linking:

Name: mongocrypt
Description: The libmongocrypt client-side field level encryption library.
Version: 1.7.2
Requires:
prefix=/usr
includedir=${prefix}/include/mongocrypt
libdir=${prefix}/lib/lib64
Libs: ${libdir}/libmongocrypt-static.a ${libdir}/libkms_message-static.a -L${libdir} -lbson-static-1.0 /usr/lib/librt.a /usr/lib/libm.a -lssl -lcrypto -ldl -pthread
Clfags: -I/${includedir} -I/usr/include/kms_message -I/usr/include/libbson-1.0 -DBSON_STATIC

Comment by Divjot Arora (Inactive) [ 08/Sep/20 ]

ido@ido50.net Thanks for confirming! I've updated the ticket title and changed it to "Improvement" to reflect the work that needs to be done. I don't think there's anything for us to do until the upstream libmongocrypt work is done, so I'm going to move this out of "Investigating" and we'll pick it up again when we're unblocked.

Comment by Ido Perlmuter [ 08/Sep/20 ]

Sounds good to me, thanks!

Comment by Divjot Arora (Inactive) [ 08/Sep/20 ]

ido@ido50.net After some more investigation, I believe this requires changes in libmongocrypt packaging. The pkg-config output for libmongocrypt-static doesn't output all of the required flags to link against libbson, which causes "undefined reference" errors when building the driver. I've brought this up to the team that maintains libmongocrypt and they're going to investigate further.

Once we resolve these build issues, I'm hoping to add a new cse-static build tag for the driver. If enabled, the driver will check for pkg-config: libmongocrypt-static rather than pkg-config: libmongocrypt in this line: https://github.com/mongodb/mongo-go-driver/blob/33c65f5f06d172aa2425932e76c75932ea5f58dd/x/mongo/driver/mongocrypt/mongocrypt.go#L11. Would that be a reasonable way to enable this feature?

Comment by Divjot Arora (Inactive) [ 03/Sep/20 ]

Hi ido@ido50.net,

Apologies for the delay on our end. Just wanted to let you know that I'm actively looking into this. As you can imagine, this is a pretty complex issue and is difficult to prototype/test because we're dependent on an external libmongocrypt library so even small changes require some time to test. I'm talking to the team that maintains libmongocrypt and is responsible for building and distributing the packages for it to see if we can figure out a path forward. I should have an update for you next week.

– Divjot

Generated at Thu Feb 08 08:37:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.