[SERVER-47006] Implement a downgrade floor between 4.4 and 4.2 Created: 20/Mar/20  Updated: 29/Oct/23  Resolved: 02/Apr/20

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.2.6, 4.4.0-rc0, 4.7.0

Type: Improvement Priority: Major - P3
Reporter: Alexander Gorrod Assignee: Daniel Gottlieb (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-46784 Make jstests/multiVersion/keystring_i... Closed
Related
is related to WT-5890 Bump log/WT version to facilitate upg... Closed
is related to WT-5892 Bump log/WT version to facilitate dow... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2020-03-23, Execution Team 2020-04-06
Participants:
Linked BF Score: 50

 Description   

We have realized that we need a downgrade floor, because it's possible that multi-shard multi-document transactions will generate data that existing versions of MongoDB 4.2 won't be able to read.

We are planning to increment the WiredTiger compatibility version in MongoDB 4.2 from 3.2.2 to 3.3.0 in WT-5892. In order for the downgrade floor to work we need MongoDB to also set some specific compatibility version strings in wiredtiger_open.

We need to set compatibility=(release=3.2) to make sure that MongoDB 4.2 keeps producing log files in a format that older releases understand.

We need to set compatibility=(requires_max=3.3) to allow the 4.2 release after this change is included to open a database generated by 4.4 (after it's had a clean shutdown).

There is a related but orthogonal change in WT-5890 - where we are changing it so that WiredTiger in MongoDB 4.4 will use compatibility version 10.0, and generate log files that can't be read by older versions of MongoDB. On clean shutdown the compatibility version is decreased so that it generates log files that correspond to compatibility=3.3.



 Comments   
Comment by Githook User [ 02/Apr/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-47006: Implement a downgrade floor between 4.4 and 4.2 SERVER-47068: Remove repair lock file when WT salvage refused to work on incompatible data files.
Branch: master
https://github.com/mongodb/mongo/commit/907fc7a809ae947f24395033027c68de9c7ae415

Comment by Githook User [ 30/Mar/20 ]

Author:

{'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'username': 'dgottlieb'}

Message: SERVER-47006: Correct a comment typo in MDB/WT versioning.
Branch: v4.4
https://github.com/mongodb/mongo/commit/eee72639cc0b1fe1e5f6877da44fbff054bd647c

Comment by Githook User [ 25/Mar/20 ]

Author:

{'email': 'daniel.gottlieb@mongodb.com', 'name': 'Daniel Gottlieb', 'username': 'dgottlieb'}

Message: SERVER-47006 Implement a downgrade floor between 4.4 and 4.2
Branch: v4.2
https://github.com/mongodb/mongo/commit/7300c441a25b886b9bca5e15e6be7fa7e451a85e

Comment by Githook User [ 24/Mar/20 ]

Author:

{'email': 'daniel.gottlieb@mongodb.com', 'name': 'Daniel Gottlieb', 'username': 'dgottlieb'}

Message: SERVER-47006 Implement a downgrade floor between 4.4 and 4.2
Branch: v4.4
https://github.com/mongodb/mongo/commit/70c19c844d348774290fdadb9f9bbb0deb5301ad

Comment by Alexander Gorrod [ 22/Mar/20 ]

Thanks for the excellent write-up daniel.gottlieb - that matches the plan is a much clearer explanation than I've been able to muster.

Comment by Susan LoVerso [ 20/Mar/20 ]

To make sure I got this right, I'll state my interpretation/understanding (which I'm extending to include 4.0 so I understand the bigger picture)

daniel.gottlieb thank you for this clear and concise statement, it speaks the language I understand and helps me a lot in the reviews I'm doing.

Comment by Daniel Gottlieb (Inactive) [ 20/Mar/20 ]

To make sure I got this right, I'll state my interpretation/understanding (which I'm extending to include 4.0 so I understand the bigger picture):

  • MongoDB 4.0 runs in WT 3.1.0 and writes log version 3.
  • MongoDB 4.0 shutting down in FCV 3.6 will leave WT in a 3.0.0 state with log version 2
  • MongoDB 4.2.6+ will receive a backport that allows it to read log versions 4
  • In the absence of 4.2.6 explicitly calling wiredtiger_open with compatibility=(release=3.2), it would write out data in 3.3.0/log version 4
  • MongoDB 4.4 will run in WT 10 and write log version 5
  • MongoDB 4.4 shutting down in FCV 4.2 will write out data in WT 3.3.0/log version 4
  • MongoDB 4.4 shutting down in FCV 4.4 will leave data in WT 10.0/log version 5
  • Compatibility version 3.3/log version 4 will only produced by MongoDB 4.4 that was shut down in FCV 4.2
  • For MongoDB 4.2.6, running with compatibility=(release=3.2) will result in an unconfigured WT_CONNECTION::close to write out 3.2/log version 3 files. An explicit compatibility=(release=3.2) is not needed to downgrade to 4.2.4-.
  • MongoDB 4.2.6 will also technically understand WT compatibility version 10.0/log version 5? (thus the requirement to require_max=3.3.0)
Comment by Alexander Gorrod [ 20/Mar/20 ]

milkie and daniel.gottlieb would you mind taking a look at this? It's blocking some upgrade downgrade work, so I'd appreciate it if we could get it addressed quickly.

Generated at Thu Feb 08 05:13:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.