Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-46975

Provide lzma compressed packages

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Minor - P4 Minor - P4
    • None
    • Affects Version/s: 4.3.4
    • Component/s: Packaging
    • Labels:
    • Build

      The size of the 4.3.4 release package (mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz) is 120 mb. When testing the driver, we download this package in each evergreen build. This is quite speedy in the office (and I imagine in evergreen) but when I am working from home, my transfer rate is closer to 3 mb/s rather than 30 mb/s that I get in the office. At 3 mb/s it takes 40 seconds to download the package.

      As an experiment I compressed the tarball with `xz` and `xz -9`, using the following commands:

      cat mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz|gunzip|xz >mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz     
      cat mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz|gunzip|xz -9 >mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz

      The results are:

      -rw-rw-r-- 1 w w  50676376 Mar 18 23:49 mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz
      -rw-rw-r-- 1 w w 127709954 Mar  6 17:26 mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz
      -rw-rw-r-- 1 w w  85076852 Mar 18 23:45 mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz

      The package compressed with `xz -9` is 40% of the size of gzip-compressed package. A 70 mb reduction in size would mean a 23 second reduction in download time at 3 mb/s.

      Extraction times for the three packages are as follows:

      tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4.tgz  2.20s user 0.16s system 108% cpu 2.181 total
      tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4.txz  5.47s user 0.29s system 105% cpu 5.459 total
      tar tf mongodb-linux-x86_64-enterprise-rhel62-4.3.4-9.txz  3.60s user 0.31s system 107% cpu 3.643 total

      Interestingly it takes less time to extract at compression level 9 than at the default level. It only takes 1.5 more seconds to extract lzma level 9-compressed package compared to the gzip package.

      I think it will be beneficial if the server provided lzma-compressed packages in addition to the gzip-compressed ones.

            Unassigned Unassigned
            oleg.pudeyev@mongodb.com Oleg Pudeyev (Inactive)
            0 Vote for this issue
            4 Start watching this issue