[SERVER-38145] Extend upgradeCheckAllDBs() and/or upgradeCheck() to look for indexes not compliant with version 3.4 index definition rules Created: 15/Nov/18 Updated: 09/May/19 Resolved: 09/May/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | William Byrne III | Assignee: | Kelsey Schubert |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||
| Issue Links: |
|
||||
| Participants: | |||||
| Case: | (copied to CRM) | ||||
| Description |
|
Version 3.4 introduced stricter validation of index specifications. However, the new code is only run when an index is recreated, meaning the following scenario is all too possible:
To enable DBA's to detect and fix index definitions that do not comply with the stricter rules in version 3.4 and later, it would be nice if the upgradeCheck utilities (or some other new utility) existed that would examine all indexes for a collection/database/cluster and report on those that cannot be upgraded. It would be better to discover and repair them in planned outage windows than in the midst of an already unplanned outage. Many of these boolean fields in version 3.2 index keys are called unique, sparse and background, meaning they were almost certainly intended to be index options and not part of the key, so the index is not created as intended either, and it is in the customer's interest to fix that too. |
| Comments |
| Comment by Kelsey Schubert [ 09/May/19 ] | ||||||||||||
|
Including this feature inside of MongoDB is expensive, and may require minimum floors before upgrading. Instead, we're looking into external tools that behave similar to the script William provided which would address the same user pain without concern about which particular version they are are on before upgrading. | ||||||||||||
| Comment by William Byrne III [ 07/Jan/19 ] | ||||||||||||
|
The attached script has been replaced with one that has comments: index34.js The side effects if the non-compliant key field comes from index options mistakenly added to the key is as follows:
The impact of indexes with these mistakes in their definition is mostly small. Once created, the index that was intended to be built in the background will probably be used by the Query Planner in exactly the way the index with the correct definition would be, and will return the correct results. This is because the intended key is most commonly an [index prefix](https://docs.mongodb.com/v3.4/core/index-compound/#prefixes) of the actual key. Similarly, the non-sparse index will assist queries correctly. All three options specification errors will waste space, as the extra non-existent field name at the end of the key will be indexed as a null for every entry. An index intended to be sparse will waste more space as it will index all the documents with null key field values. The most serious potential impact is from uniqueness not being enforced as expected, as that means duplicate documents would be loaded without error. |