Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-31060

Two phase drops with too long MMAPv1 index names must generate dropIndexes operations before drop operation

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Replication
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Sprint:
      Repl 2017-10-02, Repl 2017-10-23

      Description

      Collection drops are two phase, so a collection is renamed to a temporary namespace before it is physically dropped. On MMAP, there is a hard limit on namespace lengths, so we will drop any indexes with names that would be too long following the drop collection rename (SERVER-29747). Currently, we are physically dropping the offending indexes and then renaming the dropped collection, however, we are generating oplog entries in the reverse order. That is, we actually log the 'drop' collection operation first, and then any 'dropIndexes' operations.

      In theory, this should break two phase drop behavior on MMAP when there are offending indexes (see jstests/replsets/drop_collections_two_phase_long_index_names.js), since, when secondaries try to apply the operations, they apply the drop collection operation first, attempting a rename of a collection that would create index names that are too long, resulting in a fatal assertion. It seems, that, inadvertently, we avoided this case by also not properly checking writesAreReplicated in the dropCollectionEvenIfSystem function. On primary, we should perform the index drops and collection rename, but the secondaries should simply apply the drop indexes oplog operations as normal operations and then apply the drop collection op.

      In general, it seems incorrect to be logging "non-sensical" oplog entries i.e. a dropIndex on a collection that no longer exists. This may manifest as a bug when trying to rollback a drop collection operation that drops indexes in this manner, since you would be rolling back a dropIndex on a collection that doesn't exist. However, it might also be avoided there since rollback via refetch takes some freedom in re-ordering operations that it is reverting.

      Dumped primary oplog from run of drop_collections_two_phase_long_index_names.js below. You can see the 'dropIndexes' operation appear after the 'drop' operation on collection uuid '932ed914-d0d5-44ce-bba7-5c4a2b05adf0':

      Dumping the latest 20 documents that match { } from the oplog oplog.rs of williams-ubuntu:20010
      {  "ts" : Timestamp(1505236565, 4),  "t" : NumberLong(1),  "h" : NumberLong("-5285471193402663181"),  "v" : 2,  "op" : "c",  "ns" : "drop_collection_two_phase_long_index_names.$cmd",  "ui" : UUID("932ed914-d0d5-44ce-bba7-5c4a2b05adf0"),  "o2" : {  "v" : 2,  "key" : {  "a" : 1 },  "name" : "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",  "ns" : "drop_collection_two_phase_long_index_names.collToD
      rop" },  "wall" : ISODate("2017-09-12T17:16:05.374Z"),  "o" : {  "dropIndexes" : "collToDrop",  "index" : "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" } }
      {  "ts" : Timestamp(1505236565, 3),  "t" : NumberLong(1),  "h" : NumberLong("2127366147023087314"),  "v" : 2,  "op" : "c",  "ns" : "drop_collection_two_phase_long_index_names.$cmd",  "ui" : UUID("932ed914-d0d5-44ce-bba7-5c4a2b05adf0"),  "wall" : ISODate("2017-09-12T17:16:05.373Z"),  "o" : {  "drop" : "collToDrop" } }
      {  "ts" : Timestamp(1505236565, 2),  "t" : NumberLong(1),  "h" : NumberLong("2818049393288337882"),  "v" : 2,  "op" : "c",  "ns" : "drop_collection_two_phase_long_index_names.$cmd",  "ui" : UUID("932ed914-d0d5-44ce-bba7-5c4a2b05adf0"),  "wall" : ISODate("2017-09-12T17:16:05.040Z"),  "o" : {  "createIndexes" : "collToDrop",  "v" : 2,  "key" : {  "b" : 1 },  "name" : "short_name" } }
      {  "ts" : Timestamp(1505236565, 1),  "t" : NumberLong(1),  "h" : NumberLong("-6587097313745957566"),  "v" : 2,  "op" : "c",  "ns" : "drop_collection_two_phase_long_index_names.$cmd",  "ui" : UUID("932ed914-d0d5-44ce-bba7-5c4a2b05adf0"),  "wall" : ISODate("2017-09-12T17:16:05.033Z"),  "o" : {  "createIndexes" : "collToDrop",  "v" : 2,  "key" : {  "a" : 1 },  "name" : "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
      aaaaaaaaaaaaaaaaa" } }
      {  "ts" : Timestamp(1505236559, 1),  "t" : NumberLong(1),  "h" : NumberLong("4104972344915778556"),  "v" : 2,  "op" : "c",  "ns" : "drop_collection_two_phase_long_index_names.$cmd",  "ui" : UUID("932ed914-d0d5-44ce-bba7-5c4a2b05adf0"),  "wall" : ISODate("2017-09-12T17:15:59.080Z"),  "o" : {  "create" : "collToDrop",  "idIndex" : {  "v" : 2,  "key" : {  "_id" : 1 },  "name" : "_id_",  "ns" : "drop_collection_two_phase_long
      _index_names.collToDrop" } } }
      {  "ts" : Timestamp(1505236557, 2),  "t" : NumberLong(1),  "h" : NumberLong("8357852249975499374"),  "v" : 2,  "op" : "n",  "ns" : "",  "wall" : ISODate("2017-09-12T17:15:57.268Z"),  "o" : {  "msg" : "Reconfig set",  "version" : 2 } }
      {  "ts" : Timestamp(1505236557, 1),  "t" : NumberLong(1),  "h" : NumberLong("-647256883118635236"),  "v" : 2,  "op" : "c",  "ns" : "config.$cmd",  "ui" : UUID("91cd72bd-d995-456f-a1b4-6a97d0d440e7"),  "wall" : ISODate("2017-09-12T17:15:57.187Z"),  "o" : {  "create" : "transactions",  "idIndex" : {  "v" : 2,  "key" : {  "_id" : 1 },  "name" : "_id_",  "ns" : "config.transactions" } } }
      {  "ts" : Timestamp(1505236556, 1),  "t" : NumberLong(1),  "h" : NumberLong("5993574424827143666"),  "v" : 2,  "op" : "n",  "ns" : "",  "wall" : ISODate("2017-09-12T17:15:56.913Z"),  "o" : {  "msg" : "new primary" } }
      {  "ts" : Timestamp(1505236554, 1),  "h" : NumberLong("1918102913157968804"),  "v" : 2,  "op" : "n",  "ns" : "",  "o" : {  "msg" : "initiating set" } }
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              benety.goh Benety Goh
              Reporter:
              william.schultz William Schultz (Inactive)
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: