[SERVER-18899] Enable fsyncLock on WiredTiger to allow backups via copy Created: 09/Jun/15  Updated: 28/May/18  Resolved: 25/Sep/15

Status: Closed
Project: Core Server
Component/s: WiredTiger
Affects Version/s: None
Fix Version/s: 3.1.9

Type: Improvement Priority: Major - P3
Reporter: Daniel Pasette (Inactive) Assignee: David Hows
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by WT-1973 MongoDB changes for WiredTiger 2.7.0 Closed
is depended on by SERVER-17557 Display warning when running db.fsync... Closed
is depended on by SERVER-20637 Backup/restore test - enable fsyncLoc... Closed
Documented
is documented by DOCS-7661 File system snapshots for backups und... Closed
is documented by DOCS-6273 fsyncLock now supported with WiredTiger Closed
Related
is related to SERVER-20389 Fix backup_restore.js test performing... Closed
Backwards Compatibility: Major Change
Participants:

 Description   

This could be enabled by taking a backup cursor during lock and releasing it during unlock.

See: http://source.wiredtiger.com/develop/backup.html



 Comments   
Comment by David Hows [ 25/Sep/15 ]

This change adds 2 new calls into the storage API, beginBackup and endBackup. These are required calls now when running the fsync command

Comment by Githook User [ 25/Sep/15 ]

Author:

{u'username': u'daveh86', u'name': u'David Hows', u'email': u'howsdav@gmail.com'}

Message: SERVER-18899 - Add beginBackup and endBackup to storage API
Branch: master
https://github.com/mongodb/mongo/commit/e5bd1ecd48d78900bafd64022200f94eb7be24c7

Comment by David Hows [ 15/Sep/15 ]

Hey redbeard0531,

Would appreciate your input here. As a consequence of making the changes we discussed around returning CommandNotSupported by default, there are a large number of test failures for rocksdb.

Looking over those failures, they mostly seem to come from places where we are using fsyncLock to block writes, rather than guarantee filesystem consistency (replication tests where we block writes to secondary's are a common example). There are about 10~ test suites which would need to be modified in order to stop these errors.

Wanted your input on how best to proceed, currently I see the following as options:

  1. Leave the tests broken until we fix the issue in mongo-rocks
  2. Modify all of theses tests to skip the relevant part when fsync returns CommandNotSupported
Comment by Githook User [ 14/Sep/15 ]

Author:

{u'username': u'daveh86', u'name': u'David Hows', u'email': u'howsdav@gmail.com'}

Message: Revert "SERVER-18899 - Add beginBackup and endBackup to storage API"

This reverts commit e1d7404eae3a2c33fa4f15105b931f17d0c3d502.
Branch: master
https://github.com/mongodb/mongo/commit/3325cd25cd45ca7bdf856f62a740497ba7eedf36

Comment by Githook User [ 14/Sep/15 ]

Author:

{u'username': u'daveh86', u'name': u'David Hows', u'email': u'howsdav@gmail.com'}

Message: SERVER-18899 - Add beginBackup and endBackup to storage API
Branch: master
https://github.com/mongodb/mongo/commit/e1d7404eae3a2c33fa4f15105b931f17d0c3d502

Comment by Martin Bligh [ 24/Aug/15 ]

michael.cahill , david.hows - if it's fairly quick and simple, do you want to just write a change and send it across for review? If it's more complex and is going to risk a lot of time, can discuss it first.

Comment by Michael Cahill (Inactive) [ 24/Aug/15 ]

dan@10gen.com, this just needs beginFSyncLock / endFSyncLock hooks in the storage engine API (or better would be beginBackup / endBackup). The WiredTiger implementation of those methods would just be to open/close a backup cursor.

I can ask daveh86 to make the changes but the storage engine API side will need input from martin.bligh, redbeard0531 and/or geert.bosch.

Comment by Githook User [ 08/Jul/15 ]

Author:

{u'username': u'sueloverso', u'name': u'sueloverso', u'email': u'sue@mongodb.com'}

Message: Merge pull request #2053 from wiredtiger/testsuite-refactor-amend

SERVER-18899 followup to abstract copying a WT home directory into a function.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/3b38f011ab1c34d6a420105592f2bf866bde14c5

Comment by Githook User [ 08/Jul/15 ]

Author:

{u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@ddanderson.com'}

Message: SERVER-18899 followup to abstract copying a WT home directory into a function,
and create two callers to the function.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/718f244fdd07c82e5390960af3a0c0155ccba390

Comment by Githook User [ 01/Jul/15 ]

Author:

{u'username': u'michaelcahill', u'name': u'Michael Cahill', u'email': u'michael.cahill@mongodb.com'}

Message: Merge pull request #2027 from wiredtiger/fsynclock-test

SERVER-18899 Add unit test to simulate fsyncLock.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/d04083d3402501b06301196e409e4bf034aeee35

Comment by Githook User [ 01/Jul/15 ]

Author:

{u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

Message: SERVER-18899 Add Window-specific function.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/105f7d990e8dccd54e76fb3b045a6bd4408e63d6

Comment by Githook User [ 01/Jul/15 ]

Author:

{u'username': u'sueloverso', u'name': u'Susan LoVerso', u'email': u'sue@wiredtiger.com'}

Message: SERVER-18899 Add unit test to simulate fsyncLock.
Branch: develop
https://github.com/wiredtiger/wiredtiger/commit/26ac5aea07aedb09e9b5aa58ca20d07468dede42

Generated at Thu Feb 08 03:49:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.