Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-37798

Crash after dropping a unique index.

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 4.0.2
    • Component/s: Index Maintenance
    • Labels:
      None
    • ALL
    • Hide

       

      Fresh MongoDB 4.0.2 with wired tiger, ssl and authentication (user/pwd)

      Create collection with some indices, one of them unique.

      Insert data in this collection from pymongo 3.7.1

      Drop the unique index from pymongo 3.7.1

      MongoDB crashes (even when both data and index are owned by mongod:mongod and permissions are 700).

      Restart the server and "version is too new for this mongod" arises.

       

       

      Show
        Fresh MongoDB 4.0.2 with wired tiger, ssl and authentication (user/pwd) Create collection with some indices, one of them unique. Insert data in this collection from pymongo 3.7.1 Drop the unique index from pymongo 3.7.1 MongoDB crashes (even when both data and index are owned by mongod:mongod and permissions are 700). Restart the server and "version is too new for this mongod" arises.    

       

      In a fresh MongoDB 4.0.2 installation with wiredtiger, ssl and user/pwd authentication a unique index is dropped and the server crashes (with a permission denied error but both data and indices folder's owner is mongodb:mongodb and permissions are 700).

      018-10-29T12:17:43.640+0100 I ACCESS [conn2072] Successfully authenticated as principal twins on twitter_2014
2018-10-29T12:17:43.642+0100 I COMMAND [conn2072] CMD: dropIndexes DBNAME.COLLNAME
2018-10-29T12:17:43.780+0100 E STORAGE [conn2072] WiredTiger error (13) [1540811863:779697][16638:0x7f63132d8700], WT_SESSION.drop: __posix_fs_remove, 216: /var/lib/mongodb/index/3-8048314178193759520.wt: file-remove: unlink: Permission denied Raw: [1540811863:779697][16638:0x7f63132d8700], WT_SESSION.drop: __posix_fs_remove, 216: /var/lib/mongodb/index/3-8048314178193759520.wt: file-remove: unlink: Permission denied
2018-10-29T12:17:43.780+0100 E STORAGE [conn2072] WiredTiger error (13) [1540811863:780261][16638:0x7f63132d8700], WT_SESSION.drop: __meta_track_apply, 156: metadata remove dropped file index/3-8048314178193759520.wt: Permission denied Raw: [1540811863:780261][16638:0x7f63132d8700], WT_SESSION.drop: __meta_track_apply, 156: metadata remove dropped file index/3-8048314178193759520.wt: Permission denied
2018-10-29T12:17:43.780+0100 E STORAGE [conn2072] WiredTiger error (13) [1540811863:780286][16638:0x7f63132d8700], WT_SESSION.drop: __wt_meta_track_off, 352: failed to apply or unroll all tracked operations: Permission denied Raw: [1540811863:780286][16638:0x7f63132d8700], WT_SESSION.drop: __wt_meta_track_off, 352: failed to apply or unroll all tracked operations: Permission denied
2018-10-29T12:17:43.780+0100 E STORAGE [conn2072] WiredTiger error (-31804) [1540811863:780302][16638:0x7f63132d8700], WT_SESSION.drop: __wt_panic, 523: the process must exit and restart: WT_PANIC: WiredTiger library panic Raw: [1540811863:780302][16638:0x7f63132d8700], WT_SESSION.drop: __wt_panic, 523: the process must exit and restart: WT_PANIC: WiredTiger library panic
2018-10-29T12:17:43.780+0100 F - [conn2072] Fatal Assertion 50853 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 401
2018-10-29T12:17:43.780+0100 F - [conn1978] Fatal Assertion 28559 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 62
2018-10-29T12:17:43.780+0100 F - [conn35] Fatal Assertion 28559 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 62

      When trying to restart the service, another error arises: 

      2018-10-29T12:30:14.590+0100 I STORAGE [initandlisten] Placing a marker at optime Oct 29 12:08:58:811 2018-10-29T12:30:18.023+0100 E STORAGE [initandlisten] WiredTiger error (-31803) [1540812618:23278][25646:0x7f4985626a00], file:WiredTiger.wt, WT_CURSOR.next: __schema_create_collapse, 111: metadata information for source configuration "colgroup:index/0-6878827104047333897" not found: WT_NOTFOUND: item not found Raw: [1540812618:23278][25646:0x7f4985626a00], file:WiredTiger.wt, WT_CURSOR.next: __schema_create_collapse, 111: metadata information for source configuration "colgroup:index/0-6878827104047333897" not found: WT_NOTFOUND: item not found 2018-10-29T12:30:18.177+0100 I STORAGE [initandlisten] Expected index data is missing, rebuilding. Collection: DBNAME.COLLNAME Index: id_1 2018-10-29T12:30:18.179+0100 I STORAGE [initandlisten] not rebuilding interrupted indexes 2018-10-29T12:30:20.127+0100 F - [initandlisten] Fatal assertion 28579 UnsupportedFormat: Unable to find metadata for table:index/3-8048314178193759520 Index: {name: id_1, ns: DBNAME.COLLNAME} - version too new for this mongod. See http://dochub.mongodb.org/core/4.2-downgrade-index for detailed instructions on how to handle this error. at src/mongo/db/storage/wiredtiger/wiredtiger_index.cpp 294 2018-10-29T12:30:20.127+0100 F - [initandlisten] ***aborting after fassert() failure

      As this is a fresh install, it cannot be an upgrade/downgrade issue, how is it possible to have inconsistent index version in this case?

      Server is finally restarted by removing the corrupted index file.

       

       

       

       

        1. drop_index.py
          0.3 kB
        2. ls.txt
          71 kB
        3. mongod.log.2018-10-29T12-07-50
          16 kB

            Assignee:
            daniel.hatcher@mongodb.com Danny Hatcher (Inactive)
            Reporter:
            atugores Antònia Tugores
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: