Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-18

make mongodump multi-threaded / parallel

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 2.8.0-rc0
    • Legacy C++ Implementation
    • mongodump
    • None
    • MCI 2.8.0-RC0

    Description

      Mongodump is single threaded and therefor very slow to dump on large databases.

      The simplest approach to making mongodump parallelized is to assign one thread per collection (up to some user-defined or adaptively arrived at limit). It would be easy to overload either source or client machine by using multiple threads.

      Going further, it may be possible to partition individual collections by range. This could be tricky in sharded clusters.

      Attachments

        Activity

          People

            kyle.erf Kyle Erf
            ian@mongodb.com Ian Whalen (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: