[SERVER-22116] Memory limit of MigrationSourceManager is too big Created: 11/Jan/16  Updated: 08/Feb/23  Resolved: 09/Dec/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: 3.0.8, 3.2.0
Fix Version/s: 3.4.23

Type: Bug Priority: Major - P3
Reporter: 아나 하리 Assignee: [DO NOT USE] Backlog - Sharding Team
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-22656 Componentize Migration{Source,Dest}Ma... Closed
Assigned Teams:
Sharding
Backwards Compatibility: Fully Compatible
Operating System: ALL
Participants:

 Description   

MigrationSourceManger is caching the document id of modification during moving chunk.

But I think the threshold of memory usage is too big.
According to the source code, current limit is 5TB. Move chunk command is checking whether the memory usage is greater than 500*1024*1024, but MigrationSourceManager::mbUsed() already return MB not Bytes. So eventually MoveChunkCommand will be aborted when memory usage is greater than 5TB.

https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/move_chunk_command.cpp#L397
if (shardingState->migrationSourceManager()->mbUsed() > (500 * 1024 * 1024)) {
// This is too much memory for us to use so we're going to abort the migration
...

https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_source_manager.cpp#L557
long long MigrationSourceManager::mbUsed() const

{ stdx::lock_guard<stdx::mutex> lk(_mutex); return _memoryUsed / (1024 * 1024); }

 Comments   
Comment by Ramon Fernandez Marina [ 11/Jan/16 ]

Thanks for your report matt.lee, sending it to the Sharding team for evaluation.

Generated at Thu Feb 08 03:59:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.