[SERVER-32994] Create a script to rebuild unique indexes in right format after a downgrade Created: 30/Jan/18 Updated: 29/Oct/23 Resolved: 06/Aug/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.2, 4.1.2 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Neha Khatri | Assignee: | Luke Chen |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | nonnyc, storage-engines | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Backport Requested: |
v4.0
|
||||||||||||||||||||
| Sprint: | Storage Non-NYC 2018-04-09, Storage Engines 2018-07-30, Storage Engines 2018-08-13 | ||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
When a MongoDB 4.2 database is downgraded to FCV=4.0, the existing unique indexes in data format 11 and 12 should continue to operate in this format. Create a script that users can run that will identify and rebuild all of these indexes in the older format of 6 and 8. Coordinate with docs team regarding publishing of this script. |
| Comments |
| Comment by Luke Chen [ 21/Oct/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi kay.kim, The docs changes in the link look good to me. To answer your questions, yes the script will rebuild all unique indexes, no matter they are initially created with 4.0 or 4.2 binary. There could be ways to tell which ones are with 4.0-format that could be excluded from the recreation process (like what milkie mentioned above), but it's likely not trivial/straightforward to implement and communicate to users. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 06/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'username': 'lukech', 'name': 'Luke Chen', 'email': 'luke.chen@mongodb.com'}Message: (cherry picked from commit e49247580c76ccdd4013ff0357e03d0b1c95cbb6) | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Luke Chen [ 06/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
alexander.gorrod and kay.kim, I just pushed the change of updating error message with the new dochub.mongodb.org/core/4.2-downgrade-index link. Will backport the change to v4.0 to get the desired effect - error out by pointing to the new link while starting data files with 4.2-format unique indexes using a 4.0 binary. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Githook User [ 06/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author: {'name': 'Luke Chen', 'email': 'luke.chen@mongodb.com', 'username': 'lukech'}Message: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 02/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Thanks kay.kim - that's very useful! luke.chen could you work on the change to add that message into MongoDB? | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Kim (Inactive) [ 02/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
alexander.gorrod Will do. In the meantime, I can give you a dochub link dochub.mongodb.org/core/4.2-downgrade-index which you can add to MongoDB and you all can finish up on your side since you don't have to worry about where it points. That link just redirects to whatever page we tell it to. So until the upcoming downgrade page exists, I can have it point to some page (like the unique index page or whatnot). When the 4.2 downgrade page/section exists, I'll have it point to the correct page but you don't need to change the link on your side. The dochub link remains the same, we on our side just tell it to point somewhere else. I haven't created the dochub.mongodb.org/core/4.2-downgrade-index link yet – so it will 404. But let me know if you want to proceed like that. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 02/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
luke.chen I just re-opened this ticket, and put it on the backlog. kay.kim it would be great if you can let us know when the docs are done - so we can add a message to MongoDB that includes a link to the documentation. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Kay Kim (Inactive) [ 01/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
luke.chen – your script looks great. Should be enough. Thanks! | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Luke Chen [ 01/Aug/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hi kay.kim, I got this ticket that may need documentation update regarding 4.2 --> 4.0 downgrade. Could you please have a check and let me know if you need more information? Thanks. A new set of internal formats (i.e. 11 and 12) will be implemented for unique indexes in 4.2 (via PM-934), replacing the old set of internal unique index formats (i.e. 6 and 8) of 4.0 (and prior releases). The corresponding external unique index versions are v:1 and v:2 which are visible and customisable to MongoDB users. When users try to downgrade mongod from 4.2 to 4.0, after setting FCV to 4.0, all unique indexes (both v:1 and v:2) need to be dropped and recreated with the same external version, so that all recreated unique indexes have the old set of formats (i.e. 6 and 8) that 4.0 is able to handle. Below script (code snippet) can be used to drop and recreate unique indexes during 4.2 --> 4.0 downgrading. It had been verified in our test environment with the co-existence of unique index external/internal version permutations (i.e. v:1/6, v:2/8, v:1/11, v:2/12) and a 4.2 --> 4.0 downgrade procedure.
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 30/Jul/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
An example of such a script and associated documentation from a previous backward breaking change is: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Alexander Gorrod [ 07/May/18 ] | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Guess is 5 days to complete. |