[SERVER-46975] Provide lzma compressed packages Created: 19/Mar/20  Updated: 02/Feb/24

Status: Backlog
Project: Core Server
Component/s: Packaging
Affects Version/s: 4.3.4
Fix Version/s: None

Type: Improvement Priority: Minor - P4
Reporter: Oleg Pudeyev (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Build
Participants:

 Description   

The size of the 4.3.4 release package (mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz) is 120 mb. When testing the driver, we download this package in each evergreen build. This is quite speedy in the office (and I imagine in evergreen) but when I am working from home, my transfer rate is closer to 3 mb/s rather than 30 mb/s that I get in the office. At 3 mb/s it takes 40 seconds to download the package.

As an experiment I compressed the tarball with `xz` and `xz -9`, using the following commands:

cat mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz|gunzip|xz >mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz     
cat mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz|gunzip|xz -9 >mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz

The results are:

-rw-rw-r-- 1 w w  50676376 Mar 18 23:49 mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz
-rw-rw-r-- 1 w w 127709954 Mar  6 17:26 mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz
-rw-rw-r-- 1 w w  85076852 Mar 18 23:45 mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz

The package compressed with `xz -9` is 40% of the size of gzip-compressed package. A 70 mb reduction in size would mean a 23 second reduction in download time at 3 mb/s.

Extraction times for the three packages are as follows:

tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz  2.20s user 0.16s system 108% cpu 2.181 total
tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz  5.47s user 0.29s system 105% cpu 5.459 total
tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz  3.60s user 0.31s system 107% cpu 3.643 total

Interestingly it takes less time to extract at compression level 9 than at the default level. It only takes 1.5 more seconds to extract lzma level 9-compressed package compared to the gzip package.

I think it will be beneficial if the server provided lzma-compressed packages in addition to the gzip-compressed ones.



 Comments   
Comment by Steve Gross [ 28/Aug/23 ]

Per backlog review w/ alex.neben@mongodb.com : this is still a good idea. Keeping open.

Comment by Brian Samek [ 15/Apr/22 ]

This was incorrectly assigned to Evergreen's backlog user. Removed, and flagged for scheduling.

Generated at Thu Feb 08 05:12:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.