[DOCS-12429] Incorrect description of behavior of fsyncLock with WT Created: 06/Feb/19  Updated: 30/Oct/23  Resolved: 11/Feb/19

Status: Closed
Project: Documentation
Component/s: manual
Affects Version/s: None
Fix Version/s: Server_Docs_20231030

Type: Bug Priority: Major - P3
Reporter: Bruce Lucas (Inactive) Assignee: Ravind Kumar (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-37814 db.fsyncLock() do not prevent writes ... Closed
Participants:
Days since reply: 5 years, 2 days ago
Epic Link: DOCSP-1769

 Description   

Description

In our description of fsyncLock we say that fsyncLock ensures "that the data files do not change", but I don't believe this is true: it only ensures that they don't change in a way that will corrupt a copy taken while the instance is fsyncLock'ed.

Scope of changes

Impact to Other Docs

  • Establish definition for fsynclock with better examples of what it does/does not do
  • Document and backport.

MVP (Work and Date)

Resources (Scope or Design Docs, Invision, etc.)



 Comments   
Comment by Githook User [ 11/Feb/19 ]

Author:

{'name': 'rkumar-mongo', 'email': 'ravind.kumar@mongodb.com'}

Message: DOCS-12429: Clarify fsync/db.fsyncLock data file lock behavior
Branch: v3.4
https://github.com/mongodb/docs/commit/d9aee3d9176dd2adecbe7c61b9b68b54a4d80642

Comment by Ravind Kumar (Inactive) [ 11/Feb/19 ]

Changes published to master/v4.0 -> v3.2.

Comment by Githook User [ 11/Feb/19 ]

Author:

{'name': 'rkumar-mongo', 'email': 'ravind.kumar@mongodb.com'}

Message: DOCS-12429: Clarify fsync/db.fsyncLock data file lock behavior
Branch: v4.0
https://github.com/mongodb/docs/commit/10778a2ae1656da089c7b3321161c4d38f0da5c5

Comment by Githook User [ 11/Feb/19 ]

Author:

{'name': 'rkumar-mongo', 'email': 'ravind.kumar@mongodb.com'}

Message: DOCS-12429: Clarify fsync/db.fsyncLock data file lock behavior
Branch: v3.2
https://github.com/mongodb/docs/commit/30a9472376c4a4fa5c703e6baeea19e6dd6010a1

Comment by Githook User [ 11/Feb/19 ]

Author:

{'name': 'rkumar-mongo', 'email': 'ravind.kumar@mongodb.com'}

Message: DOCS-12429: Clarify fsync/db.fsyncLock data file lock behavior
Branch: v3.6
https://github.com/mongodb/docs/commit/e454095c47cfa8bdf93d3ebb834d3d75aa0290c7

Comment by Githook User [ 11/Feb/19 ]

Author:

{'name': 'rkumar-mongo', 'email': 'ravind.kumar@mongodb.com'}

Message: DOCS-12429: Clarify fsync/db.fsyncLock data file lock behavior
Branch: master
https://github.com/mongodb/docs/commit/57e34e7d86d556297e1330904a66f39be45b749a

Comment by Alexander Gorrod [ 07/Feb/19 ]

From a documentation perspective I think what we need to say is that under WT fsyncLock does not prevent the data files from changing, but it does ensure that it is safe to copy the files even though they may change during the process.

That sounds like an accurate description to me.

Comment by Bruce Lucas (Inactive) [ 06/Feb/19 ]

OK. From a documentation perspective I think what we need to say is that under WT fsyncLock does not prevent the data files from changing, but it does ensure that it is safe to copy the files even though they may change during the process.

The importance of the distinction is illustrated by SERVER-37814: some backup programs, including tar, will notice that the files have changed and complain, and users see this as a problem because it differs from the documentation. (Some backup programs might even fail in this case, I think, and if so that would make fsyncLock problematic, but that's a different topic.)

Comment by Daniel Gottlieb (Inactive) [ 06/Feb/19 ]

there is something you need to do via the WT API to ensure correct hot backup conditions

As a clarification, I wouldn't consider these hot backups. The system is up, but it is not accepting any writes. There's a 4.2 (completed) project that allows file system copies while still accepting writes. It uses the same WT backup cursor technique under the hood, just without taking a global X lock.

Comment by Daniel Gottlieb (Inactive) [ 06/Feb/19 ]

Yes, we create a backup cursor on WT. That call is what satisfies the second property.

Comment by Daniel Gottlieb (Inactive) [ 06/Feb/19 ]

That's getting rather in the weeds. I don't think it's useful to describe "which bytes can change on disk", nor do I think that's a stable contract. I may have misread the request. Logical means nothing the user should care about. It's the data the user interfaces with (the stuff they insert/read/update/delete/aggregate/map reduce along with the collections/indexes/constraints they make).

I'd prefer if the documentation stipulated two things:

  • The logical data (as in, what can be queried from a client) will not change.
  • The files on disk can be copied elsewhere. Starting up a mongod on those files (using flags that go through all recovery processes, which we can provide more information on what that means) will have data that's indistinguishable from the node which is fsyncLocked.

It's probably best to caveat data down to "what users put into the database" and not internal writes. Examples of internal writes include update to <database>.system.* collections and replication related collections in the local database. I don't think* any of those can be occur while the system is fsyncLocked, but stating so seems like an unnecessary thing to guarantee.

Comment by Ravind Kumar (Inactive) [ 06/Feb/19 ]

Can we define or maybe spitball some examples of 'logical changes' that would not occur? That I think might be more useful for users as far as conceptualizing exactly what fsynclock does and does not do. Or does journal flushes + checkpoints more or less cover what fsynclock does allow.

 

Generated at Thu Feb 08 08:05:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.