-
Type: Bug
-
Resolution: Won't Fix
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Sharding EMEA
-
ALL
-
(copied to CRM)
ChunkWritesTracker objects are instantiated on shards to track the number of bytes that are being written on every owned chunk; the metric is then used to trigger the chunk splitter when a sufficient number (threshold probably to review) of bytes have been written to a chunk.
Currently, the bytesWritten counter:
- Is incremented when a new document is inserted ✅
- Is incremented when a document gets updated ❌ wrong because the increment/decrement should take into account the bytes diff between the old document and its updated version.
- Does not change on deletes ❌ wrong because the counter must take into account removed documents and be decremented.
Objective of this ticket is correcting the metrics in order to avoid scheduling chunk splitting tasks too much or too early. Currently, the two main problems we can incur under some conditions are:
- Unnecessary load on the CPU (a lot of noop split tasks).
- Eagerness to split (in case of the new auto-splitter, continuously try to splitt a chunk so that when it gets exactly at 133% maxChunkSize it is split right away in two chunks of 67% maxChunkSize).