[SERVER-26287] createIndexes should confirm that index options are valid in respect to the underlying collection Created: 23/Sep/16 Updated: 25/Apr/22 Resolved: 19/Nov/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance |
| Affects Version/s: | None |
| Fix Version/s: | 5.2.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | James Wahlin | Assignee: | Gabriel Marks |
| Resolution: | Done | Votes: | 0 |
| Labels: | execution_intern, neweng | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||||||||||||
| Steps To Reproduce: |
|
||||||||||||||||||||||||||||||||||||
| Sprint: | Execution Team 2021-11-01, Execution Team 2021-11-15, Execution Team 2021-11-29 | ||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||
| Description |
|
As reported in We should audit which index options are invalid for specific collection options and add validation so that createIndexes will fail if specified. |
| Comments |
| Comment by Githook User [ 18/Nov/21 ] |
|
Author: {'name': 'Gabriel Marks', 'email': 'gabriel.marks@mongodb.com', 'username': 'marksg07'}Message: |
| Comment by Eric Milkie [ 05/Nov/21 ] |
|
Yes, or just try it with the option, catch the new error we're adding, and then retry without the option. Then you wouldn't have to poll every collection state. |
| Comment by Gregory Noma [ 01/Nov/21 ] |
|
Yes that would be the ideal solution. One thing to potentially note about this is that it would require checking the state of the collection to see if it is capped, rather than just examining the index options themselves. |
| Comment by Eric Milkie [ 29/Oct/21 ] |
|
If possible, the ideal outcome is for mongomirror to omit the expireAfterSeconds regardless of source and destination version, rather than failing. The failing option would require a user to then drop and rebuild that index without TTL (if they still needed the index) on their source system. |
| Comment by Tim Fogarty [ 29/Oct/21 ] |
|
I added mirror/tools to downstream changes so we have a ticket open for this now. |
| Comment by Tim Fogarty [ 29/Oct/21 ] |
|
Yes we only use listIndexes to replicate indexes between clusters. What you outlined sounds like a good plan to allow round-trips with same-version migrations. We do support mixed versions, so that will still be an issue in that case. I think the most similar change we've had in the past was dropping support for geoHaystack indexes in 5.0. For that we detect any geoHaystack indexes when we list indexes so mongomirror will fail before starting the migration. I think that's what we should do here too. If we're going to fail, we prefer to fail early. |
| Comment by Gregory Noma [ 27/Oct/21 ] |
|
The only work left to do for this ticket should be to disallow creating TTL indexes on capped collections, which our documentation explicitly calls out as being disallowed. Note that while creating a TTL index on a capped collection currently succeeds, the index won't actually be TTL. tim.fogarty does mongomirror (or any other tools) use only listIndexes to take indexes from one node and build them on another, or is there any other source of index information it uses? My idea is to exclude the expireAfterSeconds field from any index specs when running listIndexes on a capped collection, since those indexes aren't really TTL anyway. That way tools like mongomirror can still round-trip existing "TTL" indexes on capped collections without any changes, at least as long as both the source and destination nodes are on a version that includes this change. Any thoughts on this idea? Would the mixed-version scenario still be an issue? (I'm not sure exactly what we support for mongomirror.) I guess the more general question is how we've dealt with changes like this in the past. |
| Comment by Eric Milkie [ 21/Oct/21 ] |
|
Is this fix going to allow tools like mongomirror the ability to still build indexes that fail this check, or is there downstream work for all the tools to avoid bad index specs? |