[SERVER-71771] Defragmenter should cap chunk size estimation Created: 01/Dec/22  Updated: 29/Oct/23  Resolved: 16/Jan/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 6.0.3, 6.1.0-rc4, 6.2.0-rc2
Fix Version/s: 6.3.0-rc0, 6.0.5

Type: Bug Priority: Major - P3
Reporter: Tommaso Tocci Assignee: Paolo Polato
Resolution: Fixed Votes: 0
Labels: collection-defrag-bugs
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-71756 Defragmentation policy should issue d... Closed
is related to SERVER-71759 dataSize command doesn't yield Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.2, v6.1, v6.0
Sprint: Sharding EMEA 2022-12-12, Sharding EMEA 2022-12-26, Sharding EMEA 2023-01-09, Sharding EMEA 2023-01-23
Participants:
Linked BF Score: 170

 Description   

The defragmenter in phase1 ("MergeAndMesure") issue dataSize commands to the shard in order to estimate the size of the chunks.

Phase2 use this estimation to:
 - ignore all the chunks bigger than a certain threshold
 - order and process all the remaining chunks from the smaller to the bigger.

Since we need an estimation of the data size just for smaller chunks we can avoid calculating it  for the bigger ones (they will be ignored by phase 2 in any case).

In order to so we could send the dataSize command with the maxSize parameter so that if the shard reach this limit it will stop scanning the index and will set the maxReached flag in the reply.



 Comments   
Comment by Githook User [ 13/Jan/23 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Fix overflow error in Collection Defragmentation Algorithm

(cherry picked from commit daa6fb3e8fdae969996e211a2e83e1b35ef80992)
Branch: v6.0
https://github.com/mongodb/mongo/commit/9f75b518e341c3e867418f16e06647ffa58bc82f

Comment by Githook User [ 13/Jan/23 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Fix overflow error in Collection Defragmentation Algorithm
Branch: master
https://github.com/mongodb/mongo/commit/daa6fb3e8fdae969996e211a2e83e1b35ef80992

Comment by Githook User [ 05/Jan/23 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Defragmenter should cap chunk size estimation
Branch: v6.0
https://github.com/mongodb/mongo/commit/3d7cb234e07f7d6e027f0c355fc6bd8aa9e1f965

Comment by Githook User [ 05/Jan/23 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Defragmenter should cap chunk size estimation
Branch: master
https://github.com/mongodb/mongo/commit/9ef944976420211a063f5b502e38991f0490eb65

Comment by Githook User [ 04/Jan/23 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: Revert "SERVER-71771 Defragmenter should cap chunk size estimation"

This reverts commit d0d9e95c07df276f4fa3cef07982803d689da5c7.
Branch: master
https://github.com/mongodb/mongo/commit/1eb33d31a32bb7d1a1a18e07c78239b90a9dd5d3

Comment by Githook User [ 29/Dec/22 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: Revert "SERVER-71771 Defragmenter should cap chunk size estimation"

This reverts commit 43221bcda3d39d58b8ef648185c63d3840791b4f.
Branch: v6.0
https://github.com/mongodb/mongo/commit/6d8ebdbe466580ef90af022f5946c93608d627ad

Comment by Githook User [ 28/Dec/22 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Defragmenter should cap chunk size estimation

(cherry picked from commit d0d9e95c07df276f4fa3cef07982803d689da5c7)
Branch: v6.0
https://github.com/mongodb/mongo/commit/1cc097075cc9531cf24a43ed249012d719b2b4fe

Comment by Githook User [ 28/Dec/22 ]

Author:

{'name': 'Paolo Polato', 'email': 'paolo.polato@mongodb.com', 'username': 'ppolato'}

Message: SERVER-71771 Defragmenter should cap chunk size estimation
Branch: master
https://github.com/mongodb/mongo/commit/d0d9e95c07df276f4fa3cef07982803d689da5c7

Generated at Thu Feb 08 06:19:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.