[SERVER-20176] Deletes with j:true slower on WT than MMAPv1 Created: 28/Aug/15  Updated: 31/Aug/23  Resolved: 19/Sep/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: 3.0.6, 3.1.7
Fix Version/s: 3.0.7, 3.1.9

Type: Bug Priority: Major - P3
Reporter: Andre de Frere Assignee: Geert Bosch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on WT-2104 add support for flushing the transact... Closed
Duplicate
is duplicated by SERVER-20576 Calling WiredTigerRecoveryUnit::waitU... Closed
is duplicated by SERVER-18746 ensure WT write-ahead log flush is as... Closed
Related
related to SERVER-20617 wt_nojournal_toggle.js failing interm... Closed
related to SERVER-80432 waitForSession can miss JournalListen... Closed
related to SERVER-17489 in bulk ops, only mark last operation... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Sprint: Quint 9 09/18/15
Participants:

 Description   

j:true is expectedly slower than not, but WT is much slower than MMAPv1:

MMAPv1

> for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
WriteResult({ "nInserted" : 1 })
> var start=new Date();db.test.remove( { } );duration=new Date() - start
70
> for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
WriteResult({ "nInserted" : 1 })
> var start=new Date();db.test.remove( { }, { writeConcern : {j:true}} );duration=new Date() - start
94

WT

> for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
WriteResult({ "nInserted" : 1 })
> var start=new Date();db.test.remove( { } );duration=new Date() - start
189
> for(x=0;x<10000;x++){db.test.insert( { x : x } ) }
WriteResult({ "nInserted" : 1 })
> var start=new Date();db.test.remove( { }, { writeConcern : {j:true}} );duration=new Date() - start
95113



 Comments   
Comment by Igor Canadi [ 03/Oct/15 ]

Cool. This should fix it: https://reviews.facebook.net/D48087

Comment by Geert Bosch [ 02/Oct/15 ]

Yes, that's true. We have some cases where we don't really have anything to write, but still need to wait until the last write was durable. So in essence, we'd be doing an empty unit of work without any writes, but we still need to wait for any writes that happened previously to be durable.

I think this is unavoidable, as we avoid doing a write if no actual data changes, but the semantics of w:1 require that all previous writes be durable...

-Geert

Comment by Igor Canadi [ 02/Oct/15 ]

Hey Geert, seems like we no longer call goingToAwaitDurable() before waitUntilDurable(). In tests, I see an invariant:

Invariant failure _sync src/mongo/db/modules/rocksdb/src/rocks_recovery_unit.cpp 261

Source: https://logkeeper.mongodb.org/build/560ea1af90413021b1b50d3d/test/560ea225be07c4135fb508c0

This is the code for invariant(), it's pretty straight-forward: https://github.com/mongodb-partners/mongo-rocks/blob/master/src/rocks_recovery_unit.cpp#L261

I'll try to debug where this happens, but if you can offer any insights, I'd be very grateful.

Comment by Githook User [ 25/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: For writeConcern j:1 only sync once at end of operation

(cherry picked from commit 37b328aeda5696ff21c86b1258708fe5bede3a16)
(cherry picked from commit 617a1eb81610f2cc00b6aac06e408dfc17246f04)
(cherry picked from commit 38f6c23578438a2d67c0846f9623740513829c33)
(cherry picked from commit 2033a39cabed03c0efc598e439c5e1eec8767243)
(cherry picked from commit f86c9a1dfd585947762296d232391ba0f1bd304a)

Conflicts:
src/mongo/db/storage/recovery_unit.h
src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.cpp
src/mongo/db/storage/wiredtiger/wiredtiger_recovery_unit.h
src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.cpp
src/mongo/db/storage/wiredtiger/wiredtiger_snapshot_manager.h
Branch: v3.0
https://github.com/mongodb/mongo/commit/88e5b19dc436208effc324daff5c54c7bd261803

Comment by Githook User [ 24/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: Add extra check for j:1 durability
Branch: master
https://github.com/mongodb/mongo/commit/f86c9a1dfd585947762296d232391ba0f1bd304a

Comment by Githook User [ 24/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: Clarify contract of waitUntilDurable
Branch: master
https://github.com/mongodb/mongo/commit/38f6c23578438a2d67c0846f9623740513829c33

Comment by Githook User [ 23/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: Don't assume visibility implies durability in test
Branch: master
https://github.com/mongodb/mongo/commit/2033a39cabed03c0efc598e439c5e1eec8767243

Comment by Githook User [ 23/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: Use a checkpoint instead of log_flush for --nojournal
Branch: master
https://github.com/mongodb/mongo/commit/617a1eb81610f2cc00b6aac06e408dfc17246f04

Comment by Githook User [ 19/Sep/15 ]

Author:

{u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

Message: SERVER-20176: For writeConcern j:1 only sync once at end of operation
Branch: master
https://github.com/mongodb/mongo/commit/37b328aeda5696ff21c86b1258708fe5bede3a16

Generated at Thu Feb 08 03:53:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.