[SERVER-64444] listIndexes fails on invalid pre-5.0 index spec after upgrade Created: 11/Mar/22  Updated: 23/Jan/24  Resolved: 15/Apr/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.4
Fix Version/s: 6.1.0-rc0, 5.0.25, 6.0.14

Type: Bug Priority: Major - P3
Reporter: Benety Goh Assignee: Yuhong Zhang
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Related
is related to SERVER-52545 Specify input/output to listIndexes c... Closed
is related to SERVER-65549 Fix invalid index options when upgrad... Closed
is related to SERVER-65797 Remove invalid index specs in memory ... Closed
is related to SERVER-59805 Provide a way to check for and to rem... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0, v5.3, v5.0
Sprint: Execution Team 2022-04-18, Execution Team 2022-05-02
Participants:
Case:

 Description   

Invalid index options, such as specifying "sparse" as a string rather than a boolean,  may have been created in the past, before the createIndexes command started validating options more strictly.

In MongoDB 5.0, the listIndexes command more strictly validates index options, and fails when an invalid option is encountered. At this point, the only option is to drop and recreate the index.

This is a variation on the problem addressed in SERVER-59805, which detects fields that are not expected in an index spec. This ticket is about index specs with known fields but unexpected value types not allowed in the IDL definition.



 Comments   
Comment by Githook User [ 23/Jan/24 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `validate` command to detect invalid boolean index options

(cherry picked from commit 248c9c2f35e0a0b1fcf1e56c35ed2fb06682cd14)
(cherry picked from commit b9046a28ab1f3c620b139399d0d112aa03a1eba3)

GitOrigin-RevId: 7d10c32c6dbbf401a961679c52b0b4bc69e53f52
Branch: v5.0
https://github.com/mongodb/mongo/commit/d94ea5e98e5f20ff060ff9d8a9bd1b55d1873def

Comment by Githook User [ 23/Jan/24 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `collMod` command to fix invalid boolean index options

(cherry picked from commit 9b6755d018fa749405cc2a603c1c5d935a20a742)
(cherry picked from commit d8c557be9b692881a2ef796680c54936c647256a)

GitOrigin-RevId: 23f3691e54c5cad037841ad1a4cdb6653a1af2fa
Branch: v5.0
https://github.com/mongodb/mongo/commit/246ad22bffae043e8751866b82ff740ede3ff876

Comment by Githook User [ 22/Jan/24 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `validate` command to detect invalid boolean index options

(cherry picked from commit 248c9c2f35e0a0b1fcf1e56c35ed2fb06682cd14)

GitOrigin-RevId: b9046a28ab1f3c620b139399d0d112aa03a1eba3
Branch: v6.0
https://github.com/mongodb/mongo/commit/e81a1a1c255e4cab510a2ee28be2c64184e33e24

Comment by Githook User [ 22/Jan/24 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `collMod` command to fix invalid boolean index options

(cherry picked from commit 9b6755d018fa749405cc2a603c1c5d935a20a742)

GitOrigin-RevId: d8c557be9b692881a2ef796680c54936c647256a
Branch: v6.0
https://github.com/mongodb/mongo/commit/90c9c793de7afa6e68d4e015b77891981932bb4d

Comment by Githook User [ 10/Oct/22 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 SERVER-68477 Add unittests for repairing index specs

(cherry-picked from commit a59e32ba3883e46077a750d1f65f383e35ab3e12)
Branch: v5.0
https://github.com/mongodb/mongo/commit/847494d572092c8409315a94b9f3571cc6bde1a4

Comment by Githook User [ 10/Sep/22 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 SERVER-64877 Add unittests for repairing index specs

(cherry-picked from commit a59e32ba3883e46077a750d1f65f383e35ab3e12)
Branch: v6.0
https://github.com/mongodb/mongo/commit/a02d3716c2872b077d5ce6bab9e84d4d53cda53b

Comment by Githook User [ 15/Apr/22 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Add unittests for repairing and validating index specs
Branch: master
https://github.com/mongodb/mongo/commit/a59e32ba3883e46077a750d1f65f383e35ab3e12

Comment by Githook User [ 15/Apr/22 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `validate` command to detect invalid boolean index options
Branch: master
https://github.com/mongodb/mongo/commit/248c9c2f35e0a0b1fcf1e56c35ed2fb06682cd14

Comment by Githook User [ 15/Apr/22 ]

Author:

{'name': 'Yuhong Zhang', 'email': 'yuhong.zhang@mongodb.com', 'username': 'YuhongZhang98'}

Message: SERVER-64444 Extend `collMod` command to fix invalid boolean index options
Branch: master
https://github.com/mongodb/mongo/commit/9b6755d018fa749405cc2a603c1c5d935a20a742

Comment by Benety Goh [ 11/Mar/22 ]

The validation and corrective action in SERVER-59805 is based on validateIndexSpecFieldNames. The parsing of the existing index specs in the catalog can be found in this loop within the listIndexes implementation.

Couple of possible resolutions:

  • Extend the validation in SERVER-59805 to check the specs against the IDL definition, either as errors or warnings.
  • Allow listIndexes to return non-conformant legacy index specs, since these index specs are accepted by our query engine.

 

Generated at Thu Feb 08 06:00:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.