Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-845

Mongodump very slow with custom _id

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.0.4
    • Component/s: mongodump
    • None
    • Environment:
      Ubuntu 14.04 mongo 3.04 WiredTiger

      I have a collection with 75 million objects (375 bytes avg size) and one index, _id.

      mongodump of this collection takes hours.

      Some random info that may help:

      • WiredTiger, mongo 3.04
      • System has 16gb of ram, and is using provisioned I/O on amazon EC2
      • Collection uses a custom _id - a string of 22bytes
      • _id index is ~2gb in size (fits into ram)
      • CPU is very high (one core appears to be pinned) during mongodump.
      • High read disk i/o relative to write.
      • There is no other database activity; I've taken this secondary offline to test this problem out.

      Interestingly, when I do --forceTableScan, it completes in less than 15 minutes.

      Here's a representative log entry from the slow mongodump:

      2015-07-24T14:02:17.602+0000 I QUERY [conn5256] getmore LPRS.LPRSObjects cursorid:32178252446 ntoreturn:0 keyUpdates:0 writeConflicts:0 numYields:274 nreturned:12070 reslen:4194601 locks:{ Global: { acquireCount:

      { r: 550 }

      }, Database: { acquireCount:

      { r: 275 }

      }, Collection: { acquireCount:

      { r: 275 }

      } } 5550ms

        1. getIndexes.txt
          0.1 kB
        2. sampledocument.txt
          0.5 kB
        3. stats.txt
          5 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            astalwick Arlen Stalwick
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: