Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-22116

Memory limit of MigrationSourceManager is too big

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.8, 3.2.0
    • Fix Version/s: 3.4.23
    • Component/s: Sharding
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL

      Description

      MigrationSourceManger is caching the document id of modification during moving chunk.

      But I think the threshold of memory usage is too big.
      According to the source code, current limit is 5TB. Move chunk command is checking whether the memory usage is greater than 500*1024*1024, but MigrationSourceManager::mbUsed() already return MB not Bytes. So eventually MoveChunkCommand will be aborted when memory usage is greater than 5TB.

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/move_chunk_command.cpp#L397
      if (shardingState->migrationSourceManager()->mbUsed() > (500 * 1024 * 1024)) {
      // This is too much memory for us to use so we're going to abort the migration
      ...

      https://github.com/mongodb/mongo/blob/master/src/mongo/db/s/migration_source_manager.cpp#L557
      long long MigrationSourceManager::mbUsed() const

      { stdx::lock_guard<stdx::mutex> lk(_mutex); return _memoryUsed / (1024 * 1024); }

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              backlog-server-sharding Backlog - Sharding Team
              Reporter:
              matt.lee Matt SeongUck Lee
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: