[GODRIVER-836] zLib compression does not work Created: 19/Feb/19 Updated: 28/Oct/23 Resolved: 07/Mar/19 |
|
| Status: | Closed |
| Project: | Go Driver |
| Component/s: | Core API |
| Affects Version/s: | 0.3.0 |
| Fix Version/s: | 1.0.0-rc2 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Tim Fogarty | Assignee: | Tim Fogarty |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
It looks like zLib compression is broken. Running the tests with `MONGO_GO_DRIVER_COMPRESSOR=zlib` causes failures. To recreate this first set up a mongod with `--networkMessageCompressors="zlib"`. Many tests fail, but here is one example:
And the result:
I believe there are three underlying problems with the zLib compression. Firstly, `(z *ZlibCompressor) CompressBytes()` will always return an empty slice. This is because the `dest` slice is copied into a writer as the `buf` value. This `buf` is resliced as it is written to by the zLib compressor. So although the underlying array of dest is being updated, `dest`'s length is not being updated. So returning `dest` just returns a slice of zero elements. Secondly, `(z *ZlibCompressor) UncompressBytes()` will also always return an empty slice. `io.ReadFull(zlibReader, dest)` reads exactly `len(dest)` bytes from `zlibReader` into `dest`. Since `c.uncompressBuf` is passed into `UncompressBytes` as a zero element slice, zlibReader will never read anything into dest. See connection.go for context. Thirdly, the way compressors are initialized leads to race conditions for the zLib compressor. Currently, a single compressor is initialized by `topology.WithConnString()` and is used for every connection (see topology_options.go). This is fine for Snappy. But for zLib, if multiple connections use the same zLib writer at the same time, they will all be accessing and modifying the writer's internal buffers. I think initializing a new compressor per connection is probably the way to go. I'm submitting a PR to address these three issues. Would love some more eyes to ensure I haven't misunderstood anything here. |
| Comments |
| Comment by Githook User [ 07/Mar/19 ] |
|
Author: {'name': 'Tim Fogarty', 'username': 'tfogo', 'email': 'tim.fogarty@mongodb.com'}Message: Fixes the CompressBytes method of ZlibCompressor to correctly New Compressors are initialized for every connection to avoid Evergreen config is updated to run zlib tests for mongo Change-Id: I4524027a1ed85eb698a75e528b56cfe56a4d6167 |
| Comment by Tim Fogarty [ 19/Feb/19 ] |
|
Just submitted a patch here: https://review.gerrithub.io/c/mongodb/mongo-go-driver/+/445192 |