Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21027

Reduced performance of index lookups after removing documents from collection

    Details

    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:

      Description

      This ticket is a spinoff from SERVER-20876 to investigate the underlying issue that index lookups become slower after removing documents from a collection. See this comment for more details and a repro, but in summary the test

      1. creates an empty collection and index and does queries using that index
      2. inserts some number of documents into the collection
      3. removes all documents from the collection, leaving collection and index empty again
      4. repeats the same queries using the again empty index, which are observed to be significantly slower than the queries in step 1 on the new index and collection

      Referring to the stack trace in the referenced comment, the underlying cause seems to be slow WT search_near operations requiring tree walks, even though the table should be empty.

      The impact on user-level operations depends on the amount of overhead before getting to the WT layer in the path by which the query is done, but in the particular repro referenced above, declines of 15-50% were seen, depending on the number of documents inserted then removed. That path has a fair amount of overhead outside WT, so impact could be substantially higher if there are operations that do repeated lookups with low code path overhead outside WT.

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Merge pull request #2271 from wiredtiger/reverse-split-fix

          SERVER-21027 Fix reverse splits to keep the original child ref locked

          (cherry picked from commit f4d20a3)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/c819d2f9d34d8d701e986da4ea628c08239f8626

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Merge pull request #2271 from wiredtiger/reverse-split-fix SERVER-21027 Fix reverse splits to keep the original child ref locked (cherry picked from commit f4d20a3) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/c819d2f9d34d8d701e986da4ea628c08239f8626
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2278 from wiredtiger/SERVER-21027-fix

          SERVER-21027 Don't leave empty internal pages in the tree

          (cherry picked from commit ba931c1)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/7b1398a1a6ee6bd4e0624c38c5311e896a42cbfc

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2278 from wiredtiger/ SERVER-21027 -fix SERVER-21027 Don't leave empty internal pages in the tree (cherry picked from commit ba931c1) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/7b1398a1a6ee6bd4e0624c38c5311e896a42cbfc
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2278 from wiredtiger/SERVER-21027-fix

          SERVER-21027 Don't leave empty internal pages in the tree

          (cherry picked from commit ba931c1)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/7b1398a1a6ee6bd4e0624c38c5311e896a42cbfc

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2278 from wiredtiger/ SERVER-21027 -fix SERVER-21027 Don't leave empty internal pages in the tree (cherry picked from commit ba931c1) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/7b1398a1a6ee6bd4e0624c38c5311e896a42cbfc
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

          Message: Merge pull request #2330 from wiredtiger/reverse-splits-3.0_2

          SERVER-21027 Reverse split if there are many deleted pages (3.0)
          Branch: mongodb-3.0
          https://github.com/wiredtiger/wiredtiger/commit/deb2d8109ca59cc9e223fd4f5be19915b949c628

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'} Message: Merge pull request #2330 from wiredtiger/reverse-splits-3.0_2 SERVER-21027 Reverse split if there are many deleted pages (3.0) Branch: mongodb-3.0 https://github.com/wiredtiger/wiredtiger/commit/deb2d8109ca59cc9e223fd4f5be19915b949c628
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'}

          Message: Import wiredtiger-wiredtiger-mongodb-3.0.7-9-gdeb2d81.tar.gz from wiredtiger branch mongodb-3.0

          ref: cb64236..deb2d81

          deb2d81 SERVER-21027 Reverse split if there are many deleted pages (3.0)
          66a111e WT-2195 Fix a hang after giving up on a reverse split.
          7b1398a SERVER-21027 Don't leave empty internal pages in the tree
          c819d2f SERVER-21027 Fix reverse splits to keep the original child ref locked
          00dfebc SERVER-21027 Reverse split if there are many deleted pages
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/9add8acc69a119949a156b815003ecc15db75e0d

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'name': u'Ramon Fernandez', u'email': u'ramon@mongodb.com'} Message: Import wiredtiger-wiredtiger-mongodb-3.0.7-9-gdeb2d81.tar.gz from wiredtiger branch mongodb-3.0 ref: cb64236..deb2d81 deb2d81 SERVER-21027 Reverse split if there are many deleted pages (3.0) 66a111e WT-2195 Fix a hang after giving up on a reverse split. 7b1398a SERVER-21027 Don't leave empty internal pages in the tree c819d2f SERVER-21027 Fix reverse splits to keep the original child ref locked 00dfebc SERVER-21027 Reverse split if there are many deleted pages Branch: v3.0 https://github.com/mongodb/mongo/commit/9add8acc69a119949a156b815003ecc15db75e0d

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: