Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17078

show databases taking extraordinarily long with wiredTiger

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-rc6
    • Fix Version/s: 3.0.5, 3.1.5
    • Component/s: Performance, WiredTiger
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Steps To Reproduce:
      Hide

      "show databases" or doing a listdatabases command through the api/C#/java/etc.

      Show
      "show databases" or doing a listdatabases command through the api/C#/java/etc.
    • Epic Link:

      Description

      Issue Status as of Jul 14, 2015

      ISSUE SUMMARY
      When using the WiredTiger storge engine, methods to collect the size of each database for the output of the show databases operation in the mongo shell are inefficent, particularly with larger numbers of databases, collections, and indexes.

      USER IMPACT
      The show databases operation in the mongo shell can take a long time to return the listing of databases.

      WORKAROUNDS
      None.

      AFFECTED VERSIONS
      MongoDB 3.0.0 through 3.0.4.

      FIX VERSION
      The fix is included in the 3.0.5 production release.

      Original description

      When running the show databases command, with only 20 databases, we're seeing extremely long response times, typically of 60-90 seconds.

      Based on previous conversations, I'm curious if this might be related to checking database sizes and/or stating files.

      > show databases
      Admin              0.000GB
      BTest              0.003GB
      Config             0.000GB
      Customer-Fox       0.012GB
      Customer-CBS       0.040GB
      Customer-NBC       0.004GB
      Elmah              0.000GB
      MBTest             0.004GB
      Master             0.033GB
      Monitoring         0.021GB
      OCRM               0.009GB
      OCarp              0.085GB
      ODev               0.134GB
      OP                 0.041GB
      Provisions         0.002GB
      Queue              0.009GB
      local              0.000GB
      

      1. Logfile.CSV
        9.32 MB
        Chad Kreimendahl

        Issue Links

          Activity

          Hide
          paul.reed Paul Reed added a comment -

          Will "Size" become the default option and would really like the ability to be able listDatabases - without size requirements - this becoming the fastest method.

          Show
          paul.reed Paul Reed added a comment - Will "Size" become the default option and would really like the ability to be able listDatabases - without size requirements - this becoming the fastest method.
          Hide
          michael.cahill Michael Cahill added a comment -

          Paul Reed, the fast "size" mode is enabled by default for listDatabases with WiredTiger in MongoDB master here: https://github.com/mongodb/mongo/commit/c7bab6f8cfbd610ff1690cefaccb6fc83658f517

          A backport to the 3.0 branch is in progress.

          If you are requesting that listDatabases not return size information at all, can you please vote for SERVER-3181?

          Show
          michael.cahill Michael Cahill added a comment - Paul Reed , the fast "size" mode is enabled by default for listDatabases with WiredTiger in MongoDB master here: https://github.com/mongodb/mongo/commit/c7bab6f8cfbd610ff1690cefaccb6fc83658f517 A backport to the 3.0 branch is in progress. If you are requesting that listDatabases not return size information at all, can you please vote for SERVER-3181 ?
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: SERVER-17078 Add a "statistics=(size)" mode to statistics cursors that just gets the filesize without opening anything.

          (cherry picked from commit 74cea8701c115b9327523691659ed68c86c1f652)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/a0c9c24db632dfed6df03f300375ea2a275898ef

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: SERVER-17078 Add a "statistics=(size)" mode to statistics cursors that just gets the filesize without opening anything. (cherry picked from commit 74cea8701c115b9327523691659ed68c86c1f652) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/a0c9c24db632dfed6df03f300375ea2a275898ef
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: SERVER-17078 Move the statistics=(size) support into the block manager.

          (cherry picked from commit b956408d23d6abe21ce3f833c92d3ee011cfc845)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/597dddd70578fe4b6e6533fba1a76899f9e4f99f

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: SERVER-17078 Move the statistics=(size) support into the block manager. (cherry picked from commit b956408d23d6abe21ce3f833c92d3ee011cfc845) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/597dddd70578fe4b6e6533fba1a76899f9e4f99f
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: SERVER-17078 Faster WT stats for storageSize

          (cherry picked from commit c7bab6f8cfbd610ff1690cefaccb6fc83658f517)
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/9a200e2eabc59a0ac421f4cf7ec7b1c30971f9f3

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: SERVER-17078 Faster WT stats for storageSize (cherry picked from commit c7bab6f8cfbd610ff1690cefaccb6fc83658f517) Branch: v3.0 https://github.com/mongodb/mongo/commit/9a200e2eabc59a0ac421f4cf7ec7b1c30971f9f3

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              26 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: