[GODRIVER-836] zLib compression does not work Created: 19/Feb/19  Updated: 28/Oct/23  Resolved: 07/Mar/19

Status: Closed
Project: Go Driver
Component/s: Core API
Affects Version/s: 0.3.0
Fix Version/s: 1.0.0-rc2

Type: Bug Priority: Major - P3
Reporter: Tim Fogarty Assignee: Tim Fogarty
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

It looks like zLib compression is broken.

Running the tests with `MONGO_GO_DRIVER_COMPRESSOR=zlib` causes failures. To recreate this first set up a mongod with `--networkMessageCompressors="zlib"`. Many tests fail, but here is one example:

TOPOLOGY=server MONGO_GO_DRIVER_COMPRESSOR=zlib go test -v ./x/network/integration -run TestCompression

And the result:

=== RUN   TestCompression
--- FAIL: TestCompression (0.01s)
        Error Trace:    config.go:192
                        once.go:44
                        config.go:175
                        compressor_test.go:30
        Error:          Received unexpected error:
                        connection(localhost:27017[-2]) unable to decode message length: EOF
FAIL
FAIL    github.com/mongodb/mongo-go-driver/x/network/integration        0.029s

I believe there are three underlying problems with the zLib compression.

Firstly, `(z *ZlibCompressor) CompressBytes()` will always return an empty slice. This is because the `dest` slice is copied into a writer as the `buf` value. This `buf` is resliced as it is written to by the zLib compressor. So although the underlying array of dest is being updated, `dest`'s length is not being updated. So returning `dest` just returns a slice of zero elements. 

Secondly, `(z *ZlibCompressor) UncompressBytes()` will also always return an empty slice. `io.ReadFull(zlibReader, dest)` reads exactly `len(dest)` bytes from `zlibReader` into `dest`. Since `c.uncompressBuf` is passed into `UncompressBytes` as a zero element slice, zlibReader will never read anything into dest. See connection.go for context.

Thirdly, the way compressors are initialized leads to race conditions for the zLib compressor. Currently, a single compressor is initialized by `topology.WithConnString()` and is used for every connection (see topology_options.go). This is fine for Snappy. But for zLib, if multiple connections use the same zLib writer at the same time, they will all be accessing and modifying the writer's internal buffers. I think initializing a new compressor per connection is probably the way to go.

I'm submitting a PR to address these three issues. Would love some more eyes to ensure I haven't misunderstood anything here.
 



 Comments   
Comment by Githook User [ 07/Mar/19 ]

Author:

{'name': 'Tim Fogarty', 'username': 'tfogo', 'email': 'tim.fogarty@mongodb.com'}

Message: GODRIVER-836 Fix zLib compression

Fixes the CompressBytes method of ZlibCompressor to correctly
return a slice of compressed bytes. Also ensuresUncompressBytes
will correctly return a slice of uncompressed bytes. Previously
both methods would return slices of length 0.

New Compressors are initialized for every connection to avoid
race conditions with the zLib compressor.

Evergreen config is updated to run zlib tests for mongo
versions > 3.4. Snappy tests now only run for versions > 3.2.
Unit tests are added for compressors.

Change-Id: I4524027a1ed85eb698a75e528b56cfe56a4d6167
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/930d5ebf05c44dc699403343280c8010993c1ec6

Comment by Tim Fogarty [ 19/Feb/19 ]

Just submitted a patch here: https://review.gerrithub.io/c/mongodb/mongo-go-driver/+/445192

Generated at Thu Feb 08 08:35:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.