[SERVER-76602] Expose IAA compressor in MongoDB Created: 27/Apr/23  Updated: 03/Aug/23  Resolved: 03/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Jun Jin Assignee: Irwin Dolobowsky
Resolution: Won't Do Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to WT-10407 Support new compressor based on Intel... Closed
Assigned Teams:
Storage Execution
Participants:

 Description   

Based on some effort we already merged IAA compressor to WiredTiger (https://jira.mongodb.org/browse/WT-10407) now if we need to enable it in MongoDB, we need a small change in MongoDB to expose the option of IAA compressor in MongoDB data base, it will be a small PR (several line of code change) to mongodb since most of the code are in WiredTiger which is already done

Want to collect some feedback on this, should we directly submit PR ?  or should we share some perf. data on that ? 

 



 Comments   
Comment by Irwin Dolobowsky [ 03/Aug/23 ]

Thank you for your contribution.  After careful consideration we are not able to accept this submission at this time. 

 

We understand the work was already done for WT; there are features an options which WT exposes as a general purpose storage engine that MongoDB does not use.  At this time we unable to take on the additional costs to support this specific compressor.

Comment by Jun Jin [ 14/Jun/23 ]

hi keith.smith@mongodb.com we recently refined the PR on https://github.com/mongodb/mongo/pull/1543, would you pls review , if there is a need to demonstrate the performance, we would like to help. 

Comment by Jun Jin [ 15/May/23 ]

hi keith.smith@mongodb.com  the PR is submitted here: https://github.com/mongodb/mongo/pull/1543

 

Comment by Keith Smith [ 15/May/23 ]

Passing this to the Storage Execution team, since I think they own the relevant server functionality.

As I understand it, what is needed here is a bit of plumbing:

  • Provide a way for the user/application to ask to use iaa compression on a collection/index/whatever.
  • Provide the iaa compressor when opening WiredTiger.
  • Set block_compressor to iaa when creating a table.

I expect that this would be a matter of just copying what we do for zstd, or zlib today. FWIW, WT-10407 added a bit of documentation about how to use iaa in WT.

As noted in the comment below, the folks at Intel have a PR for this change.

Generated at Thu Feb 08 06:33:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.