[SERVER-58560] Implement algorithm to determine scale factor Created: 14/Jul/21  Updated: 29/Oct/23  Resolved: 18/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Henrik Edin Assignee: Henrik Edin
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-58635 Remove BufBuilder from Simple8b builder Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2021-08-09, Execution Team 2021-08-23
Participants:

 Description   

Scale factor is determined at high level in the BSON type 7 binary. When an encoder is appending doubles that need to compress we will start with scale factor 0 when starting a new Simple-8b block.

Every double that is appended need their minimal scale factor needed to be able to represent the double exactly as an integer. 

When a double is appended that needs a larger scale factor than previous doubles the encoder need to determine what can compress the most values between:

  1. Finalize current Simple-8b block and start a new block for the value that needed a larger scale factor.
  2. Re-scale previous values to the new (higher) scale factor and check if they still fit within a single Simple-8b block together with the new value.


 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 17/Aug/21 ]

Author:

{'name': 'Henrik Edin', 'email': 'henrik.edin@mongodb.com', 'username': 'henrikedin'}

Message: SERVER-58560 Double support in BSONColumnBuilder

Doubles are scaled and rounded to closest integer.
When a double is appended that needs to be scaled differently the encoder validates if flushing simple8b or re-scaling compresses best.
After writing a finalized simple8b block, the scaling is reduced to be as low as possible.
Branch: master
https://github.com/mongodb/mongo/commit/a64ddda7ade5423803695ef1dc126230bc7458f9

Generated at Thu Feb 08 05:44:51 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.