Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22116

Memory limit of MigrationSourceManager is too big

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 3.0.8, 3.2.0
    • 3.4.23
    • Sharding
    • None
    • Fully Compatible
    • ALL

    Description

      MigrationSourceManger is caching the document id of modification during moving chunk.

      But I think the threshold of memory usage is too big.
      According to the source code, current limit is 5TB. Move chunk command is checking whether the memory usage is greater than 500*1024*1024, but MigrationSourceManager::mbUsed() already return MB not Bytes. So eventually MoveChunkCommand will be aborted when memory usage is greater than 5TB.

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/move_chunk_command.cpp#L397
      if (shardingState->migrationSourceManager()->mbUsed() > (500 * 1024 * 1024)) {
      // This is too much memory for us to use so we're going to abort the migration
      ...

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_source_manager.cpp#L557
      long long MigrationSourceManager::mbUsed() const

      { stdx::lock_guard<stdx::mutex> lk(_mutex); return _memoryUsed / (1024 * 1024); }

      Attachments

        Issue Links

          Activity

            People

              backlog-server-sharding Backlog - Sharding Team
              matt.lee Matt SeongUck Lee
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: