[SERVER-33651] Mobile SE: Use full synchronous mode for SQLite writes Created: 05/Mar/18  Updated: 29/Oct/23  Resolved: 26/Jun/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.0.1, 4.1.1

Type: Improvement Priority: Major - P3
Reporter: Sulabh Mahajan Assignee: Sulabh Mahajan
Resolution: Fixed Votes: 0
Labels: nonnyc, storage-engines
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-33740 Add Evergreen task for running powerc... Closed
related to SERVER-39259 Mobile SE: Adjust SQLite Pragmas for ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Storage Non-NYC 2018-07-02
Participants:
Linked BF Score: 0
Story Points: 1

 Description   

SQLite allows some startup configuration options. The defaults should work for most of our use cases, but might still need some fine tuning. This ticket is to study the available options, and come up with any non default that might better suit us.

Also, at the conclusion of the ticket, update the design doc to specify these setting we come up with.



 Comments   
Comment by Githook User [ 10/Jul/18 ]

Author:

{'username': 'sulabhM', 'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com'}

Message: SERVER-33651 Confirm FULL synchronous mode and F_FULLFSYNC with Mobile

(cherry picked from commit 6c5046b53e8b400062b72bb556b427eff09df546)
Branch: v4.0
https://github.com/mongodb/mongo/commit/50664c0f3a502d46bb91298d254aea2220225008

Comment by Githook User [ 26/Jun/18 ]

Author:

{'username': 'kaloianm', 'name': 'Kaloian Manassiev', 'email': 'kaloian.manassiev@mongodb.com'}

Message: SERVER-33651 Fix duplicate error codes
Branch: master
https://github.com/mongodb/mongo/commit/e0e452a29e3cd4c4a8dcda6d1dd7dbdd86c684b7

Comment by Githook User [ 26/Jun/18 ]

Author:

{'username': 'sulabhM', 'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com'}

Message: SERVER-33651 Confirm FULL synchronous mode and F_FULLFSYNC with Mobile
Branch: master
https://github.com/mongodb/mongo/commit/6c5046b53e8b400062b72bb556b427eff09df546

Comment by Sulabh Mahajan [ 20/Jun/18 ]

We looked at the startup options SQLite lets us configure using sqlite3_config, there isn't anything specific that we would want to setup different than the defaults or let the user modify.

This ticket is limited to configure SQLite with PRAGMA synchronous=FULL for durability against power loss. We discussed the alternative of doing a full checkpoint with each durable write and decided to use the synchronous=FULL approach instead. It is likely to be less disruptive than checkpointing for each durable write.

Comment by Sulabh Mahajan [ 05/Mar/18 ]

Among other things to consider, we need to figure out what level to set for pragma synchronous. This would effect data loss/corruption with power failure.
For reference:
1. https://sqlite.org/wal.html#performance_considerations
2. https://jira.mongodb.org/browse/SERVER-33740?focusedCommentId=1826943&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1826943

Also, update the design as the decision is made.

Generated at Thu Feb 08 04:34:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.