[SERVER-62131] Ensure correctness of ChunkWritesTracker metrics Created: 17/Dec/21  Updated: 06/Dec/22  Resolved: 26/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: [DO NOT USE] Backlog - Sharding EMEA
Resolution: Won't Fix Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Sharding EMEA
Operating System: ALL
Participants:
Case:

 Description   

ChunkWritesTracker objects are instantiated on shards to track the number of bytes that are being written on every owned chunk; the metric is then used to trigger the chunk splitter when a sufficient number (threshold probably to review) of bytes have been written to a chunk.

Currently, the bytesWritten counter:

Objective of this ticket is correcting the metrics in order to avoid scheduling chunk splitting tasks too much or too early. Currently, the two main problems we can incur under some conditions are:

  • Unnecessary load on the CPU (a lot of noop split tasks).
  • Eagerness to split (in case of the new auto-splitter, continuously try to splitt a chunk so that when it gets exactly at 133% maxChunkSize it is split right away in two chunks of 67% maxChunkSize).


 Comments   
Comment by Pierlauro Sciarelli [ 26/Apr/22 ]

Closing as "Won't fix" because under the "No more auto splitter" project the the auto splitter is going away and subsequently the ChunkWritesTracker as well. And we realistically on't expect to target the described problem in older versions as it's not a correctness bug.

Comment by Kaloian Manassiev [ 30/Dec/21 ]

pierlauro.sciarelli, tommaso.tocci: The ChunkWritesTracker has behaved like it does since the dawn of time. Is there something that changed recently that made this behaviour worse that we have to fix it, or are you just proposing an improvement here? I am thinking that if it is the latter, we should move it under the No More Autosplitter project.

Generated at Thu Feb 08 05:54:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.