[SERVER-36228] Extend the size of TypeBits in KeyString Created: 20/Jul/18  Updated: 29/Oct/23  Resolved: 14/Aug/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 4.1 Desired
Fix Version/s: 4.1.2

Type: Improvement Priority: Major - P3
Reporter: Xiangyu Yao (Inactive) Assignee: Xiangyu Yao (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-36281 Test upgrade/downgrade behaviors for ... Closed
is depended on by SERVER-36278 Remove the 1KB index key size limit Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Storage NYC 2018-07-30, Storage NYC 2018-08-13, Storage NYC 2018-08-27
Participants:

 Description   

Currently, the maximum size of a KeyString is limited to ~1KB so it's enough that the TypeBits is a 128-byte fixed size uint8_t array. Now that we want to remove the size limit of KeyString, TypeBits needs to be extended. The work of this ticket include:

  1. Change the fixed size array to a varied size buffer in TypeBits.
  2. Extend the encode scheme to represent TypeBits more than 127 bytes. Currently, only the first element of the array is the size byte which can only represent the size of TypeBits between 0 and 127 bytes. We want to use 4 bytes to represent the size now. Detailed design can be found in the design doc.


 Comments   
Comment by Xiangyu Yao (Inactive) [ 27/Aug/18 ]

Benchmark results of storage_key_string_bm from my local machine:

---------------------------------------------------------------------
Benchmark                              Time           CPU Iterations
---------------------------------------------------------------------
BM_BSONToKeyString/V0_Int          49208 ns      49207 ns      14271   106.596MB/s   9.69052M items/s
BM_BSONToKeyString/V1_Int          49386 ns      49385 ns      14233    106.21MB/s   9.65544M items/s
BM_BSONToKeyString/V0_Double       59785 ns      59784 ns      11708   119.641MB/s   7.97606M items/s
BM_BSONToKeyString/V1_Double       51269 ns      51269 ns      13665   139.511MB/s   9.30074M items/s
BM_BSONToKeyString/V1_Decimal     131561 ns     131560 ns       5321   83.3631MB/s   3.62448M items/s
BM_BSONToKeyString/V0_String       61306 ns      61305 ns      11506   884.369MB/s   7.77809M items/s
BM_BSONToKeyString/V1_String       61780 ns      61779 ns      11422   885.118MB/s   7.71842M items/s
BM_BSONToKeyString/V0_Array      1674581 ns    1674519 ns        418   140.186MB/s   291.595k items/s
BM_BSONToKeyString/V1_Array      1444440 ns    1444424 ns        491   168.003MB/s   338.046k items/s
BM_KeyStringToBSON/V0_Int          80745 ns      80743 ns       8707   64.9615MB/s   5.90559M items/s
BM_KeyStringToBSON/V1_Int          80639 ns      80638 ns       8697   65.0466MB/s   5.91332M items/s
BM_KeyStringToBSON/V0_Double      100374 ns     100373 ns       6973   71.2599MB/s   4.75066M items/s
BM_KeyStringToBSON/V1_Double       98394 ns      98393 ns       7121    72.694MB/s   4.84627M items/s
BM_KeyStringToBSON/V1_Decimal     341280 ns     341273 ns       2059   32.1364MB/s   1.39723M items/s
BM_KeyStringToBSON/V0_String      105173 ns     105171 ns       6853   513.468MB/s   4.53393M items/s
BM_KeyStringToBSON/V1_String      101346 ns     101345 ns       6559   551.877MB/s   4.70508M items/s
BM_KeyStringToBSON/V0_Array      2294080 ns    2293998 ns        307   97.7185MB/s   212.852k items/s
BM_KeyStringToBSON/V1_Array      2114446 ns    2114420 ns        297   104.738MB/s   230.929k items/s

Comment by Githook User [ 14/Aug/18 ]

Author:

{'name': 'Xiangyu Yao', 'email': 'xiangyu.yao@mongodb.com', 'username': 'xy24'}

Message: SERVER-36228 Extend the max size of TypeBits in KeyString
Branch: master
https://github.com/mongodb/mongo/commit/3348045a336df4332ea65fbc3ce9014b6fd2786e

Generated at Thu Feb 08 04:42:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.