[SERVER-45690] Ability to customize collection compression level Created: 22/Jan/20  Updated: 06/Dec/22  Resolved: 26/May/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.2.2
Fix Version/s: None

Type: New Feature Priority: Major - P3
Reporter: Platon workaccount Assignee: Backlog - Storage Engines Team
Resolution: Duplicate Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-49405 Ability to pass --fast flag to zstd Backlog
Duplicate
is duplicated by SERVER-56508 Ability to customize Zstandard compre... Closed
Assigned Teams:
Storage Engines
Participants:

 Description   

At the moment, this setting is mentioned in the WiredTiger documentation: http://source.wiredtiger.com/mongodb-4.0/compression.html

But it cannot be applied in real practice:

coll_obj = db.create_collection("test", storageEngine={'wiredTiger': {'configString': 'block_compressor=zstd,compression_level=22'}})

Traceback (most recent call last):
 File "/home/platon/_0_Диссертация/00_Скрипты/000_ld-tools-master/backend/create_intgen_db.py", line 184, in <module>
 create_intgen_db(client)
 File "/home/platon/_0_Диссертация/00_Скрипты/000_ld-tools-master/backend/create_intgen_db.py", line 141, in create_intgen_db
 'block_compressor=zstd,compression_level=22'}})
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/database.py", line 411, in create_collection
 read_concern, session=s, **kwargs)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/collection.py", line 184, in __init__
 self.__create(kwargs, collation, session)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/collection.py", line 264, in __create
 collation=collation, session=session)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/collection.py", line 250, in _command
 user_fields=user_fields)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/pool.py", line 613, in command
 user_fields=user_fields)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/network.py", line 167, in command
 parse_write_concern_error=parse_write_concern_error)
 File "/home/platon/miniconda3/lib/python3.7/site-packages/pymongo/helpers.py", line 159, in _check_command_response
 raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: 22: Invalid argument. [1579647776:829331][874:0x7f0f8cb3e700], wiredtiger_config_validate: config_check_search, 65: unknown configuration key: 'compression_level': Invalid argument.



 Comments   
Comment by Oliver Yeh [ 02/Jul/20 ]

Looks like https://github.com/wiredtiger/wiredtiger/blob/7dfd9391862bc9a6d84868c4dc51689c45a3aacf/src/config/config_def.c doesn't have "compression_level"

Comment by Oliver Yeh [ 02/Jul/20 ]

+1 here!  According to https://github.com/facebook/zstd , zstd is now 38% faster than snappy at similar compression ratio.  We would love to be able to replace snappy with zstd if we're able to configure the compression ratio

Comment by Carl Champain (Inactive) [ 24/Jan/20 ]

Hi platon.work@gmail.com,

Thank you for the report.
I was able to successfully recreate the same error. I'm passing this ticket along to the appropriate team for further investigation.

Kind regards,
Carl
 

Generated at Thu Feb 08 05:09:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.