[SERVER-26659] Only apply stricter index key pattern validation to v:2 indexes Created: 17/Oct/16  Updated: 25/Jun/19  Resolved: 20/Oct/16

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 3.4.0-rc0
Fix Version/s: 3.4.0-rc2

Type: Bug Priority: Critical - P2
Reporter: Asya Kamsky Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to PHPLIB-447 CreateIndexes should check types of v... Backlog
related to DOCS-9049 3.4: Mention stricter createIndexes v... Closed
is related to SERVER-11064 Stricter validation of index key patt... Closed
is related to DOCS-7038 Document new restrictions on index ke... Closed
is related to SERVER-26648 tolerate bad collection metadata prod... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

Create an index using value like "true" instead of 1 or -1 in index spec. Start up 3.3/3.4 on it.

Sprint: Query 2016-10-31
Participants:

 Description   

In SERVER-11064, we made it an error for index key pattern values to be 0, NaN, or non-numeric / non-string types. Existing deployments may have v:0 and v:1 indexes which break the validation rules. As currently implemented, this will prevent upgrade to 3.4 since the validation is applied to all index versions. In order to ensure a smooth upgrade process, we should only apply the new index validation to v:2 indexes, which are new in 3.4. (v:2 indexes support collation and the decimal data type.)

Original description

Related to change made in SERVER-11064 starting up 3.4 on older mongod data directory will fail if it finds one of these technically invalid index definitions. Unfortunately the error message is not very clear as far as what the user must do to fix this:

F INDEX    [initandlisten] Found an invalid index { v: 1, key: { softDeleteTime: 1, sparse: true }, name: "softDeleteTime_1_sparse_", ns: "session_cache.session" } on the session_cache.session collection: bad index key pattern { softDeleteTime: 1, sparse: true }: Values in index key pattern cannot be of type bool. Only numbers > 0, numbers < 0, and strings are allowed.
2016-10-17T12:30:14.334-0400 I -        [initandlisten] Fatal Assertion 28782

DOCS-7038 is already open to document the changed behavior due to SERVER-11064 but I think it should be linked to directly from the error message on startup - otherwise people will not have any idea what to do and how to fix this.



 Comments   
Comment by Githook User [ 20/Oct/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-26659 only use stricter key pattern validation for v:2 or new index builds

This now follows the same rules which we will use in 3.4 for
rejecting unknown top-level options in the index spec. These
rules ensure a smooth upgrade, even in the presence of bad
index metadata produced on an older server version.
Branch: master
https://github.com/mongodb/mongo/commit/315f92405c5f5e5f4159ccbb55b0352cfe235852

Comment by Daniel Pasette (Inactive) [ 19/Oct/16 ]

david.storch That seems like the only sane approach

Comment by David Storch [ 19/Oct/16 ]

We can avoid the problem altogether by only applying the new validation to v:2 indexes. This will ensure that new indexes on fully upgraded 3.4 systems always obey the new validation rules, but also that the system can fully tolerate v:1 index specs created on old versions.

Comment by Asya Kamsky [ 17/Oct/16 ]

Also SERVER-11064 says:

if an index with an invalid key pattern is replicated from
an older version, then newer versions of mongod running in a
mixed-version replica set will fassert()

I assume that'll be the same error message that needs to be more clear.

Generated at Thu Feb 08 04:12:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.