Hi,

We are operating within a multi-cluster environment that includes primary and secondary nodes across the configuration, MongoDB routers, and multiple replicated shard clusters. Recently, we added several nodes to the shard clusters, and we have observed that rebalancing is not occurring as anticipated, resulting in significant data imbalance, causing the storage to run out of space on the rest of the clusters.

Here is the distribution of the chunks across the clusters. 

{ "_id" : "node-data1", "count" : 372246 } { "_id" : "node-data2", "count" : 372236 } { "_id" : "node-data3", "count" : 372239 } { "_id" : "node-data4", "count" : 372229 } { "_id" : "node-data5", "count" : 109849 } { "_id" : "node-data6", "count" : 109693 } { "_id" : "node-data7", "count" : 46619 } { "_id" : "node-data8", "count" : 46535 }

We are on MongoDB4.2.

I am observing a significant number of jumbo chunks for one of the largest tables, and the balancing process is proceeding very slowly.

Confirmed that the autosplitter is functioning on the shard nodes.  

 

2024-10-17T11:25:24.248+0000 I  SHARDING [ChunkSplitter-1488] request split points lookup for chunk proddb.metrics { : -3216651796548520950 } -->> { : -3216609153408564802 }
2024-10-17T11:27:43.926+0000 I  SHARDING [ChunkSplitter-1489] request split points lookup for chunk proddb.metrics { : -2441014098372422508 } -->> { : -2440993494113865685 }
2024-10-17T11:29:45.360+0000 I  SHARDING [ChunkSplitter-1490] request split points lookup for chunk proddb.metrics { : 4074468535445309800 } -->> { : 4074496847277228083 }
2024-10-17T11:32:50.063+0000 I  SHARDING [ChunkSplitter-1491] request split points lookup for chunk proddb.metrics { : -2441014098372422508 } -->> { : -2440993494113865685 }
2024-10-17T11:33:33.803+0000 I  SHARDING [ChunkSplitter-1492] request split points lookup for chunk proddb.metrics { : -3216651796548520950 } -->> { : -3216609153408564802 }

 

 

Your suggestions will be much appreciated.