[SERVER-82365] Optimize the construction of the balancer's collection distribution status histogram (2nd attempt) Created: 21/Oct/23  Updated: 15/Jan/24  Resolved: 20/Dec/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.2.25, 4.4.24, 6.0.10, 5.0.21, 7.2.0-rc0, 7.0.2, 7.1.0
Fix Version/s: 4.2.25, 5.0.23, 7.2.1, 7.3.0-rc0, 7.0.5, 6.0.13, 4.4.28, 7.0.6

Type: Improvement Priority: Major - P3
Reporter: Tommaso Tocci Assignee: Tommaso Tocci
Resolution: Fixed Votes: 1
Labels: balancer-round-perf
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-83580 Re-introduce balancer policy unittest... Closed
is depended on by SERVER-81092 Include collection namespace in balan... In Progress
Related
is related to SERVER-40459 Optimize the construction of the bala... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v7.2, v7.1, v7.0, v6.0, v5.0, v4.4, v4.2
Sprint: Sharding EMEA 2023-10-30, CAR Team 2023-11-13, CAR Team 2023-11-27, CAR Team 2023-12-11, CAR Team 2023-12-25
Participants:

 Description   

An improved version of the balancer round logic was committed in SERVER-40459  but then reverted due to SERVER-82325. So this will be the second attempt to address this performance issue.

With a considerable number of chunks, the balancer is observed to spend a large amount of time checking each chunk for belonging to a tag. This can lead to a situation where a balancer round spends most of its time finding a candidate chunk (e.g. one minute) rather than migrating a chunk. This can have a significant impact on the overall cluster balancing performance.



 Comments   
Comment by Githook User [ 14/Jan/24 ]

Author:

{'name': 'Tommaso Tocci', 'email': '58224719+toto-dev@users.noreply.github.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt) (#17653)
Branch: v7.2
https://github.com/mongodb/mongo/commit/e3098d7ab70e31b3f69d99ebd43d652cc0ef9a65

Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt)

GitOrigin-RevId: 70ab48625435c1e1717112774cd4359419f8ce70
Branch: v4.4
https://github.com/mongodb/mongo/commit/011e3ac229948230d03b9439d33806ce801b3c93

Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': '58224719+toto-dev@users.noreply.github.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt) (#16405)

GitOrigin-RevId: 330693cf875c8c870a66327db1783181f8905107
Branch: v6.0
https://github.com/mongodb/mongo/commit/2ecd91f7d9263413619812d1bae1375dcf147489

Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': '58224719+toto-dev@users.noreply.github.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt) (#17658)

GitOrigin-RevId: 439345fe588765f35dcc06ffdc79e14d8b64c047
Branch: v7.0
https://github.com/mongodb/mongo/commit/9f65f27f9849b39acb4359a01700fb70b6a935d0

Comment by Githook User [ 20/Dec/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': '58224719+toto-dev@users.noreply.github.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt) (#17653)

GitOrigin-RevId: 6790c7f2d5b387e7f7d6360ebe8a6a2a054f3c6f
Branch: master
https://github.com/mongodb/mongo/commit/306d56603d161fa331b98fc26630a345625b8506

Comment by Githook User [ 22/Nov/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt)
Branch: v4.2
https://github.com/mongodb/mongo/commit/1e19177a8141d486b39a9495eb6a5c912d661df0

Comment by Githook User [ 10/Nov/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-82365 Optimize the construction of the balancer's collection distribution status histogram (2nd attempt)
Branch: v5.0
https://github.com/mongodb/mongo/commit/ea2a1668bb4ffe53d5c8b53cf1349b235ba7b600

Comment by Tommaso Tocci [ 26/Oct/23 ]

https://github.com/10gen/mongo/pull/16405

Generated at Thu Feb 08 06:49:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.