Summary

The .NET/C# Driver supports all 3 wire compression formats (Snappy, Zstd, and Zlib) implemented by the server. Zlib is implemented using SharpCompress, a fully managed library that supports any target platform supported by .NET. Snappy and Zstd are implemented using unmanaged libraries that must be compiled for every supported platform.

We currently compile Snappy for x86/x64 Windows, x64 MacOS, and x64 Linux. We compile Zstd for x64 Windows, x64 MacOS, and x64 Linux. We do not compile either compression library for ARM64 on any operating system. Thus we cannot run natively on the Apple M1 CPU, which is an ARM64 chip. When running on .NET 5 or earlier, .NET applications run through Rosetta 2 on the Apple M1. .NET 6 includes native support for the Apple M1 and Snappy/Zstd compression will no longer work unless after upgrading to .NET 6 ARM64. (You can still run .NET 6 x64 via Rosetta 2.) See CSHARP-4101 for more information.

Snappy and Zstd for Windows are not compiled with ASLR enabled, which is a security concern for some users. We have not undertaken the work to recompile these libraries with ASLR enabled. Managed dependencies essentially implement ASLR for free due to the nature of how the managed runtime loads dependencies.

Recent improvements in managed performance including the introduction of Span<T> support have brought managed performance close to unmanaged performance even for CPU intensive operations such as compression. If we can achieve near-unmanaged performance with managed implementations of Snappy and Zstd, we would simplify our driver and packaging allowing us to run without change on any platform supported by .NET. This would notably include ARM64 targets such as Apple M1, Windows ARM64, and Linux ARM64.

This epic is to track our work in assessing the performance of managed libraries for Snappy and Zstd compared to our current unmanaged implementations. If performance is comparable, we will replace our unmanaged compression dependencies on managed ones.

Motivation

Who is the affected end user?

Users who leverage wire compression with our drivers, especially anyone using an M1 MacBook for development. Although compression is currently opt-in, DRIVERS-1335 proposes to turn it on by default. Thus supporting compression anywhere that .NET can run is becoming more desirable from both a development and support persepctive.

How does this affect the end user?

If a user opts into wire compression and attempts to run their application on an unsupported platform, they receive an error about being unable to load an unmanaged dependency. It may not be immediately obvious that the driver does not support compression on that platform and how to resolve the issue.

How likely is it that this problem or use case will occur?

This will affect every user who enables wire compression on an unsupported platform. If/when we implement DRIVERS-1335 we have the additional problem that the user can build and successfully test their application on x64 Windows or MacOS, but that application would then fail when deployed to an ARM64 Linux server. This can happen today, but the user would have to opt into wire compression. After DRIVERS-1335, wire compression would be on by default and thus this failure scenario would be more common.

If the problem does occur, what are the consequences and how severe are they?

The application would fail with an error about being unable to load an unmanaged dependency. It may not be immediately obvious that they have to disable wire compression to resolve this issue.

Is this issue urgent?

This issue has been present for awhile. While it is not urgent, DRIVERS-1335 makes it more so.

Is this ticket required by a downstream team?

No.

Is this ticket only for tests?

No.