[SERVER-34630] Change WiredTiger compatibility version setting in 3.6 release Created: 24/Apr/18  Updated: 29/Oct/23  Resolved: 18/Jun/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.6.6

Type: Task Priority: Major - P3
Reporter: Alexander Gorrod Assignee: Alexander Gorrod
Resolution: Fixed Votes: 0
Labels: SWNA, nonnyc, storage-engines
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-35271 4.0 rc0 upgrade from 3.6.5 logs "unsu... Closed
depends on WT-4098 Add new compatibility_max configurati... Closed
Gantt End to End
has to be finished together with WT-4122 Ensure compatibility downgrade cleans... Closed
Related
related to SERVER-35572 Always allow log archiving in WiredTiger Closed
is related to SERVER-32590 RTT 3.6<->4.0 upgrade/downgrade. Closed
is related to WT-4029 Bump the log file version Closed
Backwards Compatibility: Fully Compatible
Sprint: Storage Non-NYC 2018-05-07, Storage NYC 2018-05-21
Participants:
Case:

 Description   

We are introducing a log format version change in the 4.0 release, as part of recoverable rollback work. The change is being made in WT-4029.

We should update MongoDB to pass compatibility=(release=3.0) when it calls wiredtiger_open, so that users who open a database that has been used by MongoDB 4.0 and has not had a clean shutdown get a useful error message.



 Comments   
Comment by Githook User [ 18/Jun/18 ]

Author:

{'username': 'dgottlieb', 'name': 'Daniel Gottlieb', 'email': 'daniel.gottlieb@mongodb.com'}

Message: SERVER-34630 Change WiredTiger compatibility version setting in 3.6 release
Branch: v3.6
https://github.com/mongodb/mongo/commit/16909acad4d721a591d4d9b53379f8c989c682ea

Comment by Alexander Gorrod [ 18/Jun/18 ]

The expected server change here is:

diff --git a/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp b/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp
index 12cbdc2bc5..4c6254b13d 100644
--- a/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp
+++ b/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp
@@ -358,6 +358,7 @@ WiredTigerKVEngine::WiredTigerKVEngine(const std::string& canonicalName,
 
     // We are still using MongoDB's cursor cache, don't double up.
     ss << "cache_cursors=false,";
+    ss << "compatibility=(release=\"3.0\",require_max=\"3.0\"),";
 
     // The setting may have a later setting override it if not using the journal.  We make it
     // unconditional here because even nojournal may need this setting if it is a transition

Note: I edited this to remove the patch sepcification in the versions.

Comment by Alexander Gorrod [ 24/May/18 ]

michael.cahill I believe you are right. I've opened WT-4098 to track adding a requires_max option. I think we should hold off on pushing a change here until that functionality is added, and co-ordinate updating the version of WiredTiger in 3.6 to include that ticket, and pushing a SERVER change to enable using it.

Comment by Michael Cahill (Inactive) [ 17/May/18 ]

alexander.gorrod, is the change proposed here (together with SERVER-32590) sufficient to close this ticket?

Is there an additional constraint that to keep WT's mongodb-3.6 branch in sync with develop, we'll need a way to force startup to fail if the files on disk are too new? In other words, do we need require_max as well as require_min?

Comment by Daniel Gottlieb (Inactive) [ 10/May/18 ]

Daniel Gottlieb Could you confirm that the above is what you expect, and setup changes so that the correct WiredTiger versions are configured for the different MongoDB versions and configurations?

I might be missing the intent behind the "Upgraded/Downgraded" column, but omitting that results in the same WiredTiger log version, so I presume it's a benign addition. Regardless, the resulting WT log versions are correct.

Comment by Alexander Gorrod [ 09/May/18 ]

The WiredTiger side of this work is done and staged - our testing indicates that this needs more careful co-ordination than expected to avoid introducing automated test regressions.

When running the do_upgrade_downgrade.js multiversion test, it fails if we include the WiredTiger log file version change with:

[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.335+0000 2018-05-09T08:17:14.333+0000 I -        [js] shell: started program (sh26794):  /data/multiversion/mongod-3.6 --dbpath /data/db/job1/mongorunner/do_upgrade_downgrade --port 20271 --bind_ip 0.0.0.0 --nopreallocj --setParameter enableTestCommands=1 --setParameter disableLogicalSessionCacheRefresh=true --storageEngine wiredTiger --setParameter orphanCleanupDelaySecs=1 --setParameter logComponentVerbosity={"tracking":0}
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.391+0000 d20271| 2018-05-09T08:17:14.391+0000 I CONTROL  [initandlisten] MongoDB starting : pid=26794 port=20271 dbpath=/data/db/job1/mongorunner/do_upgrade_downgrade 64-bit host=ip-10-122-31-136
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.391+0000 d20271| 2018-05-09T08:17:14.391+0000 I CONTROL  [initandlisten] db version v3.6.4-80-g43a6a8a3e6
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.391+0000 d20271| 2018-05-09T08:17:14.391+0000 I CONTROL  [initandlisten] git version: 43a6a8a3e61ebb03f7b43bc587c01fe01830a6d9
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.391+0000 d20271| 2018-05-09T08:17:14.391+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.392+0000 d20271| 2018-05-09T08:17:14.392+0000 I CONTROL  [initandlisten] allocator: tcmalloc
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.392+0000 d20271| 2018-05-09T08:17:14.392+0000 I CONTROL  [initandlisten] modules: enterprise
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.392+0000 d20271| 2018-05-09T08:17:14.392+0000 I CONTROL  [initandlisten] build environment:
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.392+0000 d20271| 2018-05-09T08:17:14.392+0000 I CONTROL  [initandlisten]     distmod: rhel62
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.393+0000 d20271| 2018-05-09T08:17:14.392+0000 I CONTROL  [initandlisten]     distarch: x86_64
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.393+0000 d20271| 2018-05-09T08:17:14.393+0000 I CONTROL  [initandlisten]     target_arch: x86_64
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.393+0000 d20271| 2018-05-09T08:17:14.393+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "0.0.0.0", port: 20271 }, setParameter: { disableLogicalSessionCacheRefresh: "true", enableTestCommands: "1", logComponentVerbosity: "{"tracking":0}", orphanCleanupDelaySecs: "1" }, storage: { dbPath: "/data/db/job1/mongorunner/do_upgrade_downgrade", engine: "wiredTiger", mmapv1: { journal: { nopreallocj: true } } } }
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:14.394+0000 d20271| 2018-05-09T08:17:14.393+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3234M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),cache_cursors=false,log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.043+0000 d20271| 2018-05-09T08:17:15.043+0000 E STORAGE  [initandlisten] WiredTiger error (-31802) [1525853835:43360][26794:0x7f91a8288a80], txn-recover: unsupported WiredTiger file version: this build  only supports versions up to 2, and the file is version 3: WT_ERROR: non-specific WiredTiger error
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.044+0000 d20271| 2018-05-09T08:17:15.043+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1525853835:43863][26794:0x7f91a8288a80], txn-recover: WiredTiger is unable to read the recovery log.
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.044+0000 d20271| 2018-05-09T08:17:15.044+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1525853835:44139][26794:0x7f91a8288a80], txn-recover: This may be due to the log files being encrypted, being from an older version or due to corruption on disk
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.044+0000 d20271| 2018-05-09T08:17:15.044+0000 E STORAGE  [initandlisten] WiredTiger error (0) [1525853835:44417][26794:0x7f91a8288a80], txn-recover: You should confirm that you have opened the database with the correct options including all encryption and compression options
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.044+0000 d20271| 2018-05-09T08:17:15.044+0000 E STORAGE  [initandlisten] WiredTiger error (-31802) [1525853835:44708][26794:0x7f91a8288a80], txn-recover: Recovery failed: WT_ERROR: non-specific WiredTiger error
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.046+0000 d20271| 2018-05-09T08:17:15.046+0000 E -        [initandlisten] Assertion: 28595:-31802: WT_ERROR: non-specific WiredTiger error src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp 416
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.047+0000 d20271| 2018-05-09T08:17:15.046+0000 I STORAGE  [initandlisten] exception in initAndListen: Location28595: -31802: WT_ERROR: non-specific WiredTiger error, terminating
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.047+0000 d20271| 2018-05-09T08:17:15.047+0000 I NETWORK  [initandlisten] shutdown: going to close listening sockets...
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.047+0000 d20271| 2018-05-09T08:17:15.047+0000 I NETWORK  [initandlisten] removing socket file: /tmp/mongodb-20271.sock
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.047+0000 d20271| 2018-05-09T08:17:15.047+0000 I CONTROL  [initandlisten] now exiting
[js_test:do_upgrade_downgrade] 2018-05-09T08:17:15.047+0000 d20271| 2018-05-09T08:17:15.047+0000 I CONTROL  [initandlisten] shutting down with code:100

Which i believe is the expected failure mode. My understanding is that the following MongoDB configurations should match with WiredTiger log format versions after clean shutdown:

MongoDB binary Version FCV version Upgraded/Downgraded WiredTiger log version
4.0 4.0 No 3
4.0 3.6 No 2
4.0 3.6 Downgraded 2
4.0 4.0 Upgraded 3
3.6 3.6 No 2

daniel.gottlieb Could you confirm that the above is what you expect, and setup changes so that the correct WiredTiger versions are configured for the different MongoDB versions and configurations?

luke.chen We'll need to hold off pulling the changes for WT-4029 into MongoDB master until this is resolved.

Comment by Alexander Gorrod [ 24/Apr/18 ]

donald.anderson Would you mind picking this change up? It doesn't need to happen after WT-4029 lands - the change should be made directly in the v3.6 branch of MongoDB - we don't want it to be made in master. sueloverso is working on WT-4029 so should be able to talk through the details if you have questions.

milkie The fix version on this ticket is a bit strange - this change is targeted at the 3.6 release, but should be made prior to 4.0 being cut. Feel free to adjust the ticket setup to reflect that intended state of affairs.

Generated at Thu Feb 08 04:37:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.