[CSHARP-3612] Installing the driver using NuGet pulls compression dlls Created: 22/Apr/21  Updated: 28/Oct/23  Resolved: 11/Aug/22

Status: Closed
Project: C# Driver
Component/s: Packaging
Affects Version/s: 2.12.2
Fix Version/s: 2.18.0

Type: Improvement Priority: Major - P3
Reporter: Wan Bachtiar Assignee: Unassigned
Resolution: Fixed Votes: 5
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to MONGOCRYPT-471 Improve libmongocrypt packaging for .... Backlog
is related to CSHARP-2874 Improve Snappy dll loading and/or pac... Closed
is related to CSHARP-4271 Replace Snappy and Zstd with managed ... Closed
Backwards Compatibility: Minor Change

 Description   

Users installing MongoDB.Driver via NuGet are pulling extra `dlls`

  • Core/Compression/Snappy/lib/win/snappy32.dll
  • Core/Compression/Snappy/lib/win/snappy64.dll
  • Core/Compression/Zstandard/lib/win/libzstd.dll

Worth noting that the presence of these files in the project should not cause any issues.

However, these libraries are defined as content files and not references, they are not copied over the project during a simple NuGet restore, which may become an issue when dealing with VCS.

Original question :

https://developer.mongodb.com/community/forums/t/nuget-and-snappy-libraries-impact-on-vcs-git/10011

One of the users have kindly provided a repo example project:

https://github.com/AnturGyffrous/mongo-snappy



 Comments   
Comment by James Kovacs [ 11/Aug/22 ]

This issue has been resolved by switching to managed compression libraries (CSHARP-4271). The same problem may be present in MongoDB.Libmongocrypt. Follow MONGOCRYPT-471 for updates.

Comment by Talha Kaan Özkan [ 28/Jul/22 ]

Thank you for your quick reply, I will follow. @James Kovacs

Comment by James Kovacs [ 27/Jul/22 ]

We are actively working on CSHARP-4271, which will resolve multiple issues at the same time including this one. You can read CSHARP-4271 for a full explanation, but the short one is that we are replacing our unmanaged Snappy and Zstd dependencies with fully managed implementations. We plan to make this change in the 2.18.0 release.

Comment by Talha Kaan Özkan [ 27/Jul/22 ]

Any progress on the status of the task? can you inform us? @James Kovacs

Comment by Matteo Spreafico [ 31/Jan/22 ]

still no news?

Comment by Sasha Ostrikov [ 09/Nov/21 ]

@James Kovacs, can you give estimation about fix timing? Thanks in advance!

Comment by James Kovacs [ 05/Nov/21 ]

Hi, Matteo,

Thank you for clarifying the problem. We see now that dotnet restore does not restore the compression libraries included as content files inside the NuGet package. So removing the compression libraries from VCS is not a workable solution if you're using on-the-wire compression with the driver.

We will be re-examining our NuGet packaging strategy in a future iteration. We will take these considerations into account when we do so.

Thank you again for brining this to our attention.

Sincerely,
James

Comment by Matteo Spreafico [ 03/Nov/21 ]

Hi James,
as noted by Oleksandr that is not an option. The binaries files are included in the .csproj file, excluding them from the code repo would end in compilation errors.

regards

Comment by James Kovacs [ 02/Nov/21 ]

Hi, Matteo,

Thank you for noting the potential VCS problem. During testing with some project types, we encountered issues with references to the unmanaged libraries used for compression. The solution was to use a content files rather than references. Until we re-examine our NuGet packaging strategy, we recommend adding these unmanaged compression libraries to your .gitignore file.

Sincerely,
James

Comment by Matteo Spreafico [ 02/Nov/21 ]

I think the statement "Worth noting that the presence of these files in the project should not cause any issues" should be removed and that the severity of this case should be raised.

If you have many projects using the package (quite common in an enterprise class solution) you end up flooding the source control system with tens (maybe hundreds) of binaries files. This impact the performances of many operations (clones, comparisons, merges, etc...).

Also, you put pressure (in term of time taken and disk space used) on many of the task involved in build and release pipelines.

These are clearly issues.

m.

 

Comment by Oleksandr Seliuchenko [ 26/Oct/21 ]

I didn't remove from the csproj. Also I didn't add any changes to gitignore. I deployed my changes 2 weeks ago. Everything works.

Comment by marcos alves [ 26/Oct/21 ]

Hi,
Oleksandr Seliuchenko, did you removed from the csproj?

Comment by Oleksandr Seliuchenko [ 16/Sep/21 ]

Hi Dmitry, you suggested that we can add unmanaged binaries to .gitignore.
What should we do with these lines in the .csproj file? Can we remove these lines? Thanks.

<Content Include="Core\Compression\Snappy\lib\win\snappy32.dll" />
<Content Include="Core\Compression\Snappy\lib\win\snappy64.dll" />
<Content Include="Core\Compression\Zstandard\lib\win\libzstd.dll" />
<Content Include="mongocrypt.dll" />
<None Include="libmongocrypt.so" />
<None Include="libmongocrypt.dylib" />

Comment by Dmitry Lukyanov (Inactive) [ 27/Apr/21 ]

We agree that this behavior is not ideal, but it was required to work around other packaging issues in non-SDK projects. We suggest explicitly excluding these unmanaged binaries from VCS (using .gitignore). We will place this ticket into our backlog and will further investigate packaging improvements in the future.

Generated at Wed Feb 07 21:45:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.