[SERVER-78369] ignoreUnknownIndexOptions doesn't account for the 'weights' index field Created: 22/Jun/23  Updated: 27/Dec/23  Resolved: 20/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 6.0.6
Fix Version/s: 7.1.0-rc0, 7.0.1, 6.0.10, 5.0.21

Type: Bug Priority: Major - P3
Reporter: Stef Wang Assignee: Alberto Massari
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
causes SERVER-84434 Chunk migration may fail due to index... Open
is caused by SERVER-65797 Remove invalid index specs in memory ... Closed
Related
is related to SERVER-79645 Make CheckReplDBHash accept multi-ver... Closed
Assigned Teams:
Query Execution
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0, v6.0, v5.0
Steps To Reproduce:

1. Start a MongoDB 5.0 mongod process
2. Try to create an index that contains the field 'weights':

db.runCommand({
    createIndexes: "test",
    indexes: [{key: { key: 1, value: 1 }, name: "index_name", weights: {key: 1, name: 1}}],
    ignoreUnknownIndexOptions: true
})

This should result in an error like this:

MongoServerError: Error in specification { key: { key: 1, value: 1 }, name: "index_name", weights: { key: 1, name: 1 } } :: caused by :: Invalid index specification { key: { key: 1, value: 1 }, name: "index_name", weights: { key: 1, name: 1 } }; the field 'weights' can only be specified with text indexes

Sprint: QE 2023-07-24
Participants:
Linked BF Score: 162

 Description   

Starting in 5.0, the 'weights' field became a reserved field for text indexes. This is an issue for deployments that were upgraded from older versions to 5.0+, because they may still have non-text indexes with this 'weights' field.

This causes mongosync processes syncing data from a source cluster with the above to error out because mongosync tries to re-create the same indexes on the destination cluster that's on MongoDB 5.0. mongosync uses the ignoreUnknownIndexOptions when creating indexes, which avoids most instances of invalid index fields but this did not work for the 'weights' field (see reproduction notes below).



 Comments   
Comment by Githook User [ 22/Aug/23 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-78369 Allow index options only for the correct index types
Branch: v5.0
https://github.com/mongodb/mongo/commit/0be43c47849e154eb5ef814bd2149372478d841c

Comment by Githook User [ 21/Aug/23 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-78369 Allow index options only for the correct index types
Branch: v7.0
https://github.com/mongodb/mongo/commit/bc836934bcc8423c26cedce898c2233a8ec62e13

Comment by Githook User [ 09/Aug/23 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-78369 Allow index options only for the correct index types
Branch: v6.0
https://github.com/mongodb/mongo/commit/978895b30b06d98cb10462fd7639f4dd3e86475a

Comment by Stef Wang [ 08/Aug/23 ]

alberto.massari@mongodb.com Would the server team consider backporting this to 6.0 versions? This is currently affecting customers trying to live migrate to MongoDB Atlas 6.0.8+.

Comment by Githook User [ 19/Jul/23 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-78369 Allow index options only for the correct index types
Branch: master
https://github.com/mongodb/mongo/commit/055b10cac1f473b58332a7311bdc28a125531dc0

Comment by Githook User [ 18/Jul/23 ]

Author:

{'name': 'Gregory Wlodarek', 'email': 'gregory.wlodarek@mongodb.com', 'username': 'GWlodarek'}

Message: Revert "SERVER-78369 Allow index options only for the correct index types"

This reverts commit fb6c4e3210bfdc387210feff5de738f67fc5da72.
Branch: master
https://github.com/mongodb/mongo/commit/d87c058bce250862d2bdbc0d44c526ea4026f23a

Comment by Githook User [ 17/Jul/23 ]

Author:

{'name': 'Alberto Massari', 'email': 'alberto.massari@mongodb.com', 'username': 'albymassari'}

Message: SERVER-78369 Allow index options only for the correct index types
Branch: master
https://github.com/mongodb/mongo/commit/fb6c4e3210bfdc387210feff5de738f67fc5da72

Comment by Alberto Massari [ 10/Jul/23 ]

The fix that was supposed to ignore bad keywords used in index definition, but it has simply created a list of allowed keywords regardless of the index type. So, "weight" is not removed because it's a valid keyword for FTS indexes, even if it is used in a plain index

Generated at Thu Feb 08 06:38:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.