[DRIVERS-381] Add support for reading and writing OP_COMPRESSED Created: 10/May/17  Updated: 13/Apr/22  Resolved: 13/Apr/22

Status: Closed
Project: Drivers
Component/s: None
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Hannes Magnusson Assignee: Unassigned
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on CDRIVER-2116 Add support for OP_COMPRESSED Closed
depends on CSHARP-1991 Add support for OP_COMPRESSED Closed
depends on CXX-1356 Add support for reading and writing O... Closed
depends on GODRIVER-20 Implement Compression Specification Closed
depends on GODRIVER-49 Add support for reading and writing O... Closed
depends on JAVA-2527 OP_COMPRESSED support Closed
depends on MOTOR-162 Add support for reading and writing O... Closed
depends on NODE-1018 Add support for reading and writing O... Closed
depends on PHPC-967 Add support for reading and writing O... Closed
depends on PYTHON-1278 Support OP_COMPRESSED Closed
depends on RUBY-1216 Add support for reading and writing O... Closed
depends on RUST-54 Add support for reading and writing O... Closed
Duplicate
is duplicated by SERVER-25620 Compression of wire protocol for Clients Closed
Related
related to DOCS-11302 Document connection string options fo... Closed
Server Compat: 3.4
Start date:
Driver Compliance:
Key Status/Resolution FixVersion
PYTHON-1278 Fixed 3.7
NODE-1018 Done 3.0.0
SCALA-308 Fixed 2.2.0
JAVA-2527 Fixed 3.6.0
CSHARP-1991 Fixed 2.9.0
CXX-1356 Fixed 3.2.0-rc0
PHPC-967 Done 1.4.0-beta1, 1.4.0
RUBY-1216 Fixed 2.5.0
PERL-778 Fixed 2.0.0
CDRIVER-2116 Done 1.7.0
GODRIVER-20 Fixed 0.0.8
GODRIVER-49 Duplicate
MOTOR-162 Fixed 2.0
SWIFT-181 Won't Fix
RUST-54 Fixed 2.1.0

 Description   

MongoDB 3.4:

  • Enable compression with

    mongod --networkMessageCompressors snappy
    

  • Only supports snappy and noop
  • Compresses all responses if a supported compressor was provided in the initial isMaster (including the initial response to the isMaster!), irregardless if the request was compressed or not
  • It will compress using the first compressor configured in --networkMessageCompressors snappy,noop that was also present in the isMaster response

MongoDB 3.5-current-RC:

  • Adds support for zlib
  • snappy enabled by default (no need for --networkMessageCompressors)
  • To enable zlib or noop, use --networkMessageCompressors (remember to include snappy in the list if desired)
  • Compresses the response if the request was compressed. uses the first compressor configured in the --networkMessageCompressors (defaults to snappy if none provided) that was also present in the isMaster response

MongoDB 3.6:


OP_COMPRESSED

The new opcode looks like:

struct OP_COMPRESSED {
    struct MsgHeader {
        int32  messageLength;
        int32  requestID;
        int32  responseTo;
        int32  opCode = 2012;
    };
    int32_t  originalOpcode;
    int32_t  uncompressedSize;
    uint8_t  compressorId;
    char    *compressedMessage;
};


originalOpcode Contains the value of the wrapped opcode.
uncompressedSize The size of the deflated compressedMessage, which excludes the MsgHeader
compressorId The ID of the compressor that compressed the message
compressedMessage The opcode itself, excluding the MsgHeader


Currently supported/defined compressorIds are:

compressorId isMaster value Description
0 noop The content of the message is uncompressed. Testing Only
1 snappy The content of the message is compressed using snappy
2 zlib The content of the message is compressed using zlib
3-255 reserved Reserved for future used


Negotiation / handshake

Compression is negotiated during the MongoDB Handshake:

    {
        isMaster: 1,
        client: {}, /* See MongoDB Handshake */
        compression: ["snappy", "zlib"]
    }


New Connection String Options

compressors Comma separated values. Must warn when given unsupported compressor
zlibCompressionLevel Integer values for -1 to 9

zlibCompressionLevel meaning (informative, this is how zlib defines them)

-1 Default Compression (usually 6)
0 No compression
1 Best Speed
9 Best Compression


Blacklisted commands

The following commands MUST NOT be compressed:

  • isMaster
  • saslStart
  • saslContinue
  • getnonce
  • authenticate
  • createUser
  • updateUser
  • copydbSaslStart
  • copydbgetnonce
  • copydb


 Comments   
Comment by Githook User [ 10/May/17 ]

Author:

{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}

Message: DRIVERS-381 Add Support For Wire Compression in Drivers (snappy & zlib)
Branch: master
https://github.com/mongodb/specifications/commit/435b22098de3df4327530c2a08ec2d45399ddfb0

Generated at Thu Feb 08 08:21:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.