[SERVER-29373] Two Phase Drops: Relax index namespace length constraint under non-mmapv1 storage engines when doing collection renames for two phase drops Created: 25/May/17  Updated: 30/Oct/23  Resolved: 12/Jul/17

Status: Closed
Project: Core Server
Component/s: Storage, WiredTiger
Affects Version/s: None
Fix Version/s: 3.5.10

Type: Improvement Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-29747 Two phase drops: drop indexes before ... Closed
is related to SERVER-7720 Building an index with a too-long nam... Closed
is related to SERVER-29274 Two Phase Drops: rename collection to... Closed
is related to SERVER-29747 Two phase drops: drop indexes before ... Closed
Backwards Compatibility: Minor Change
Sprint: Repl 2017-07-31
Participants:

 Description   

Under MMAP, there's a limit on the generated index namespace in a collection due to the on-disk format. This constraint imposes a restriction on the target collection namespace when we are doing collection renames. It's likely that this constraint may not apply to WiredTiger.

This may have implications for importing databases between different storage engines and also for downgrade to previous versions of the server that still have this restriction.



 Comments   
Comment by Githook User [ 12/Jul/17 ]

Author:

{u'username': u'benety', u'name': u'Benety Goh', u'email': u'benety@mongodb.com'}

Message: SERVER-29373 relax index name constraint when doing two phase collection drops under non-mmapv1 storage engines
Branch: master
https://github.com/mongodb/mongo/commit/9bd5e1f1de06b7d6f97030036a2d03424399b5d0

Comment by Spencer Brody (Inactive) [ 08/Jun/17 ]

geert.bosch, unless we made all the drop-pending collection names be a single character long (which we can't do because we need them to be prefixed with 'system.' to avoid collisions) we would have no guarantee that any name we use will be shorter than the original collection name and thus definitely won't hit the index name limit.

I think this ticket is going to be required to make 2 phase drop and rollback work properly. We'll also need some facility we can use to ask the storage engine whether an index name is valid, that the 2 phase drop code can use to decide whether to fall back to 1 phase drops on the mmapv1 storage engine.

Comment by Eric Milkie [ 26/May/17 ]

Unfortunately, the drop oplog entry does not contain the names of the indexes in the dropped collection.

Comment by Geert Bosch [ 26/May/17 ]

It's in the drop oplog entry, no?

Comment by Eric Milkie [ 26/May/17 ]

We need the original name in order to restore it for rollback.

Comment by Geert Bosch [ 26/May/17 ]

Why does the original name need to be part of the renamed collection name? In other words, why can't you just use a random value for the name?

Comment by Benety Goh [ 25/May/17 ]

The IndexCatalog is checks the index namespace length for all storage engines. When we tried relaxing the constraint under two phase drops, we saw the following error in a test that uses long index field names:

[js_test:fsm_all_replication] 2017-05-24T13:32:57.637+0000 d20010| 2017-05-24T13:32:57.635+0000 I STORAGE  [conn1] dropCollection: db84.coll84 - renaming to drop-pending collection: db84.system.drop.1495632777i1t1.coll84 with drop optime { ts: Timestamp 1495632777000|1, t: 1 }
[js_test:fsm_all_replication] 2017-05-24T13:32:57.639+0000 d20010| 2017-05-24T13:32:57.637+0000 F INDEX    [conn1] Found an invalid index { v: 2, key: { indexed_insert_long_fieldname_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: 1.0 }, name: "indexed_insert_long_fieldname_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1", ns: "db84.system.drop.1495632777i1t1.coll84" } on the db84.system.drop.1495632777i1t1.coll84 collection: CannotCreateIndex: namespace name generated from index name "db84.system.drop.1495632777i1t1.coll84.$indexed_insert_long_fieldname_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx_1" is too long (127 byte max)
[js_test:fsm_all_replication] 2017-05-24T13:32:57.639+0000 d20010| 2017-05-24T13:32:57.637+0000 I -        [conn1] Fatal Assertion 28782 at src/mongo/db/catalog/index_catalog_impl.cpp 175

Comment by Eric Milkie [ 25/May/17 ]

Note that this relaxation should only happen for rename targets for two phase drop, as these collections will be hidden from outside observers, including other nodes in a replica set and mongodump.

Generated at Thu Feb 08 04:20:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.