[SERVER-78322] Avoid unnecessarily scanning chunks still not persisted during incremental refresh Created: 22/Jun/23  Updated: 29/Jun/23  Resolved: 29/Jun/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 6.0.6, 4.4.22, 5.0.18, 7.0.0-rc5
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Pierlauro Sciarelli Assignee: [DO NOT USE] Backlog - Sharding EMEA
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Sharding EMEA
Participants:

 Description   

On a shard, an incremental routing table refresh can start while there are still a lot of chunks that need to be persisted from a previous refresh. When this happens, almost the whole chunks vector is unnecessarily scanned in order to find the new entries.

Example

  • 1M chunks already seen by previous refresh are being persisted
  • Refresh discovers 1 new chunk
  • 1M elements will be scanned here

Solution
Since collAndChunks.changedChunks is a sorted vector, we could simply invert the direction of the loop and scan in decreasing order starting from collAndChunks.changedChunks.end() . If we consider that it is very improbable for a lot of new chunks to be discovered, this will be even more efficient than performing a binary search on the vector.


Generated at Thu Feb 08 06:38:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.