[SERVER-26648] tolerate bad collection metadata produced on version 2.4 or earlier Created: 14/Oct/16  Updated: 14/Mar/17  Resolved: 25/Oct/16

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 3.4.0-rc2

Type: Task Priority: Critical - P2
Reporter: David Storch Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
is documented by DOCS-9204 3.4 release notes compatibility chang... Closed
Related
related to SERVER-26659 Only apply stricter index key pattern... Closed
is related to SERVER-26571 cannot upgrade database created in 2.... Closed
Backwards Compatibility: Minor Change
Sprint: Query 2016-11-21
Participants:

 Description   

The collection creation path in versions 2.4.x and earlier persisted any unknown collection options in the catalog. Due to the changes SERVER-25459, recent versions of master will reject these unknown options. This causes mongod to fail during startup with fatal assertion 18523, as in the following:

2016-10-11T11:30:13.901-0400 I -        [initandlisten] Fatal assertion 18523 InvalidOptions: The field 'badOption' is not a valid collection option. Options: { badOption: "foo" } at src/mongo/db/storage/mmap_v1/mmap_v1_database_catalog_entry.cpp 876

It could also cause 3.4 secondaries replicating this metadata to shut down with a fatal assertion.

In order to make it possible for affected deployments to smoothly upgrade to 3.4, the 3.4 node must be able to tolerate extant bad metadata while also rejecting any new collection creation requests which include a bad option. Luckily, collections created on 2.4 or earlier can be identified by the fact that 2.4 also materialized the create field as the first BSONElement in the catalog metadata. Versions 2.6.x and later neither materialized the create field nor wrote unknown collection options to disk. (2.6 through 3.2 ignored unknown options but did not write them into the catalog, whereas 3.4 will reject unknown options entirely.) Therefore, only collection metadata documents which begin with the create field can include unknown options. A 3.4 node can simply ignore unknown fields during collection parsing only if the create field is present.

Going forward, we could eliminate bad collection metadata entirely by creating a command or startup flag that traverses the catalog and strips out any unknown fields. This work is out of scope for this ticket.



 Comments   
Comment by Githook User [ 25/Oct/16 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-26648 tolerate bad collection metadata produced on version 2.4 or earlier
Branch: master
https://github.com/mongodb/mongo/commit/e147caa1752b47d6bfed0f069c6081f412d2efe7

Comment by David Storch [ 24/Oct/16 ]

I've updated the title and description of this ticket to reflect our new planned approach. Note that this approach involves a fix on the master branch for version 3.4 rather than applying a change to a 3.2.x dot release.

Separately, the problem related to index key pattern validation mentioned above by asya has been resolved in SERVER-26659.

Comment by Asya Kamsky [ 17/Oct/16 ]

This is related to changes SERVER-11064 made which mean 3.4 won't start or replicate from older server/files which has invalid index spec values (like true instead of 1 for value of index key).

Generated at Thu Feb 08 04:12:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.