Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20176

Deletes with j:true slower on WT than MMAPv1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.6, 3.1.7
    • Fix Version/s: 3.0.7, 3.1.9
    • Component/s: WiredTiger
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Completed:
    • Epic Link:
    • Sprint:
      Quint 9 09/18/15

      Description

      j:true is expectedly slower than not, but WT is much slower than MMAPv1:

      MMAPv1

      > for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
      WriteResult({ "nInserted" : 1 })
      > var start=new Date();db.test.remove( { } );duration=new Date() - start
      70
      > for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
      WriteResult({ "nInserted" : 1 })
      > var start=new Date();db.test.remove( { }, { writeConcern : {j:true}} );duration=new Date() - start
      94
      

      WT

      > for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
      WriteResult({ "nInserted" : 1 })
      > var start=new Date();db.test.remove( { } );duration=new Date() - start
      189
      > for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
      WriteResult({ "nInserted" : 1 })
      > var start=new Date();db.test.remove( { }, { writeConcern : {j:true}} );duration=new Date() - start
      95113
      

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: For writeConcern j:1 only sync once at end of operation
          Branch: master
          https://github.com/mongodb/mongo/commit/37b328aeda5696ff21c86b1258708fe5bede3a16

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : For writeConcern j:1 only sync once at end of operation Branch: master https://github.com/mongodb/mongo/commit/37b328aeda5696ff21c86b1258708fe5bede3a16
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: Use a checkpoint instead of log_flush for --nojournal
          Branch: master
          https://github.com/mongodb/mongo/commit/617a1eb81610f2cc00b6aac06e408dfc17246f04

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : Use a checkpoint instead of log_flush for --nojournal Branch: master https://github.com/mongodb/mongo/commit/617a1eb81610f2cc00b6aac06e408dfc17246f04
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: Don't assume visibility implies durability in test
          Branch: master
          https://github.com/mongodb/mongo/commit/2033a39cabed03c0efc598e439c5e1eec8767243

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : Don't assume visibility implies durability in test Branch: master https://github.com/mongodb/mongo/commit/2033a39cabed03c0efc598e439c5e1eec8767243
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: Clarify contract of waitUntilDurable
          Branch: master
          https://github.com/mongodb/mongo/commit/38f6c23578438a2d67c0846f9623740513829c33

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : Clarify contract of waitUntilDurable Branch: master https://github.com/mongodb/mongo/commit/38f6c23578438a2d67c0846f9623740513829c33
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: Add extra check for j:1 durability
          Branch: master
          https://github.com/mongodb/mongo/commit/f86c9a1dfd585947762296d232391ba0f1bd304a

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : Add extra check for j:1 durability Branch: master https://github.com/mongodb/mongo/commit/f86c9a1dfd585947762296d232391ba0f1bd304a
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20176: For writeConcern j:1 only sync once at end of operation

          (cherry picked from commit 37b328aeda5696ff21c86b1258708fe5bede3a16)
          (cherry picked from commit 617a1eb81610f2cc00b6aac06e408dfc17246f04)
          (cherry picked from commit 38f6c23578438a2d67c0846f9623740513829c33)
          (cherry picked from commit 2033a39cabed03c0efc598e439c5e1eec8767243)
          (cherry picked from commit f86c9a1dfd585947762296d232391ba0f1bd304a)

          Conflicts:
          src/mongo/db/storage/recovery_unit.h
          src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp
          src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.h
          src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.cpp
          src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.h
          Branch: v3.0
          https://github.com/mongodb/mongo/commit/88e5b19dc436208effc324daff5c54c7bd261803

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20176 : For writeConcern j:1 only sync once at end of operation (cherry picked from commit 37b328aeda5696ff21c86b1258708fe5bede3a16) (cherry picked from commit 617a1eb81610f2cc00b6aac06e408dfc17246f04) (cherry picked from commit 38f6c23578438a2d67c0846f9623740513829c33) (cherry picked from commit 2033a39cabed03c0efc598e439c5e1eec8767243) (cherry picked from commit f86c9a1dfd585947762296d232391ba0f1bd304a) Conflicts: src/mongo/db/storage/recovery_unit.h src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.h src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.cpp src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.h Branch: v3.0 https://github.com/mongodb/mongo/commit/88e5b19dc436208effc324daff5c54c7bd261803
          Hide
          igor Igor Canadi added a comment -

          Hey Geert, seems like we no longer call goingToAwaitDurable() before waitUntilDurable(). In tests, I see an invariant:

          Invariant failure _sync src/mongo/db/modules/rocksdb/src/rocks_recovery_unit.cpp 261

          Source: https://logkeeper.mongodb.org/build/560ea1af90413021b1b50d3d/test/560ea225be07c4135fb508c0

          This is the code for invariant(), it's pretty straight-forward: https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_recovery_unit.cpp#L261

          I'll try to debug where this happens, but if you can offer any insights, I'd be very grateful.

          Show
          igor Igor Canadi added a comment - Hey Geert, seems like we no longer call goingToAwaitDurable() before waitUntilDurable(). In tests, I see an invariant: Invariant failure _sync src/mongo/db/modules/rocksdb/src/rocks_recovery_unit.cpp 261 Source: https://logkeeper.mongodb.org/build/560ea1af90413021b1b50d3d/test/560ea225be07c4135fb508c0 This is the code for invariant(), it's pretty straight-forward: https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_recovery_unit.cpp#L261 I'll try to debug where this happens, but if you can offer any insights, I'd be very grateful.
          Hide
          geert.bosch Geert Bosch added a comment -

          Yes, that's true. We have some cases where we don't really have anything to write, but still need to wait until the last write was durable. So in essence, we'd be doing an empty unit of work without any writes, but we still need to wait for any writes that happened previously to be durable.

          I think this is unavoidable, as we avoid doing a write if no actual data changes, but the semantics of w:1 require that all previous writes be durable...

          -Geert

          Show
          geert.bosch Geert Bosch added a comment - Yes, that's true. We have some cases where we don't really have anything to write, but still need to wait until the last write was durable. So in essence, we'd be doing an empty unit of work without any writes, but we still need to wait for any writes that happened previously to be durable. I think this is unavoidable, as we avoid doing a write if no actual data changes, but the semantics of w:1 require that all previous writes be durable... -Geert
          Hide
          igor Igor Canadi added a comment -

          Cool. This should fix it: https://reviews.facebook.net/D48087

          Show
          igor Igor Canadi added a comment - Cool. This should fix it: https://reviews.facebook.net/D48087

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile