[SERVER-53395] Use delta format for updating timeseries buckets Created: 16/Dec/20  Updated: 29/Oct/23  Resolved: 18/Dec/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.9.0

Type: Task Priority: Major - P3
Reporter: Geert Bosch Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-12-28
Participants:

 Description   

Currently we create update operations using $set to perform timeseries inserts into existing buckets. However, because $set inserts new fields into a sorted position for idempotency reasons, this means that the bucket orders elements in order like:

 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

This is a problem for unwinding and will also make it impossible to implement future optimizations to efficiency: eventually, we'd like to be able to blindly append new fields to the end of existing objects without parsing all existing fields.



 Comments   
Comment by Benety Goh [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 UpdateDriver::parse() supports delta updates for non-oplog application
Branch: master
https://github.com/mongodb/mongo/commit/a2a154c9a9aa133603d0f2797d77479a7f192ac8

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 time-series insert uses delta updates instead of classic update
Branch: master
https://github.com/mongodb/mongo/commit/a56d682b08d9a72ef54fde6d47c978b8eb54bfbc

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 time-series bucket update creates write_ops::Update, bypassing OpMsgRequest
Branch: master
https://github.com/mongodb/mongo/commit/a95a13a20fdc09c856e621588d3eb13a5048b5c8

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 bucket catalog should recalculate new fields and size delta when current bucket is full
Branch: master
https://github.com/mongodb/mongo/commit/7cf4ce85c46013c41e57cc581abe23b89028f85a

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 BucketCatalog::commit() returns new fields added
Branch: master
https://github.com/mongodb/mongo/commit/6dfd7c3ce2a9ba4f9eab989291ce03bd8074d01a

Comment by Githook User [ 18/Dec/20 ]

Author:

{'name': 'Benety Goh', 'email': 'benety@mongodb.com', 'username': 'benety'}

Message: SERVER-53395 split upsert logic for time-series into insert and update code paths
Branch: master
https://github.com/mongodb/mongo/commit/b901834c41fa9ce4a206f464f36739f271c02980

Generated at Thu Feb 08 05:30:48 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.