[SERVER-48507] Rethink wiredtiger_open contract Created: 29/May/20  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Backlog - Storage Execution Team
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by WT-6327 Skip level upgrade.js still failing a... Closed
Assigned Teams:
Storage Execution
Backport Requested:
v4.4
Participants:

 Description   

unsupported WiredTiger file version should be added here and call setWtIncompatible

Consider having Version incompatibility detected also setWtIncompatible.

New description:
See how MongoDB and WT can cooperate to better express error cases on wiredtiger_open.

MongoDB behaviors today that can hopefully be simplified:

  • MongoDB tries calling wiredtiger_open with a specific order of "compatibility versions" to learn the journal files WT discovered at startup.
    • Having a way to do this in one call would be great.
    • If multiple calls is still required, consider having a better way to suppress repeated compatibility error messages.
    • This line specifically is for WT refusing to startup on 4.0 or earlier data files.
  • Knowing whether WT has modified data files.
    • When MongoDB is running with --repair it first creates a "repair lock file". That lock file can be safely removed after repair completes.
    • However, it's typical that newer MongoDB versions can be used to repair older data (to help lessen the burden of backporting). It can be unclear if a wiredtiger_open with salvage fails because data was corrupted, or if WT refused to look at the data files because of a versioning error (e.g: 4.4 on 4.0 data files) that's outside of any compatibility constraint.


 Comments   
Comment by Susan LoVerso [ 02/Mar/21 ]

This was a lot to page back into my head after 9 months. I think the crux of the issue is that MongoDB's code referenced in this comment depends on WiredTiger performing version checks in a specific order and during that ticket changes to WT changed the order. That is where the fragility is.

I think at some point a cross-team discussion and revisit of compatibility and versions should be done to see if there is a way to make this more robust. It is probably its own project.

Some solutions could be something to do with error messages, or some way of changing the WT API so that it accepts a list of versions to check, all in one call, to remove the need for this sort of error checking in server code and repeated calls to wiredtiger_open or something else we haven't thought of.

Comment by Daniel Gottlieb (Inactive) [ 05/Jun/20 ]

Removing 4.4 required. WT has put in a change that obviated the need for this. However, the API is fragile and sue.loverso would like to revisit how MongoDB and WT communicate error cases on wiredtiger_open and how we can make the API more expressive. The goal would be for MongoDB to remove code that changes behavior based on the log messages WT emits.

Generated at Thu Feb 08 05:17:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.