Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-19274

DB fails to recover after system crash, with "No such file or directory"

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Cannot Reproduce
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.1.5
    • Component/s: Storage, WiredTiger
    • Fully Compatible
    • ALL
    • Hide

      Using the powercycle.sh script with the following arguments:

      ./powertest.sh -d https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.1.5.tgz -r pt-315 -F ~/mongodb/mongo -s 10.4.1.54  -C mpower -O output1 -S "-i /Users/jonathan/.ssh/mFi.pem admin@10.4.117.131" -R -N 10 -l 50

      Note - If you need to run this on an AWS instance you would use it as follows. The -F option is the root of the mongo source on your client.

      ./powertest.sh -d https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.1.5.tgz -r pt-315 -F ~/mongodb/mongo -s "-i <pem file> <ip addr>  -M <mongo bin dir> -R -N 10 -l 50
      Show
      Using the powercycle.sh script with the following arguments: ./powertest.sh -d https: //fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.1.5.tgz -r pt-315 -F ~/mongodb/mongo -s 10.4.1.54 -C mpower -O output1 -S "-i /Users/jonathan/.ssh/mFi.pem admin@10.4.117.131" -R -N 10 -l 50 Note - If you need to run this on an AWS instance you would use it as follows. The -F option is the root of the mongo source on your client. ./powertest.sh -d https: //fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.1.5.tgz -r pt-315 -F ~/mongodb/mongo -s "-i <pem file> <ip addr> -M <mongo bin dir> -R -N 10 -l 50
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The powercycle test created a DB that was not recoverable:

      2015-07-02T15:10:36.727-0400 I CONTROL  [main] ***** SERVER RESTARTED *****
2015-07-02T15:10:36.788-0400 W -        [initandlisten] Detected unclean shutdown - /home/jonathan/pt-315/data/wiredTiger/mongod.lock is not empty.
2015-07-02T15:10:36.788-0400 W STORAGE  [initandlisten] Recovering data from the last clean checkpoint.
2015-07-02T15:10:36.788-0400 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=8G,session_max=20000,eviction=(threads_max=4),statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2015-07-02T15:10:36.861-0400 E STORAGE  [initandlisten] WiredTiger (2) [1435864236:861392][2600:0x7f6b3b048b80], file:collection-181-5805430730046625089.wt: /home/jonathan/pt-315/data/wiredTiger/collection-181-5805430730046625089.wt: No such file or directory
2015-07-02T15:10:36.861-0400 E STORAGE  [initandlisten] WiredTiger (2) [1435864236:861464][2600:0x7f6b3b048b80], file:collection-181-5805430730046625089.wt: Operation failed during recovery: No such file or directory
2015-07-02T15:10:36.880-0400 I -        [initandlisten] Fatal assertion 28548 NoSuchKey Unable to find metadata for table:collection-175-5805430730046625089
2015-07-02T15:10:36.880-0400 I -        [initandlisten]

***aborting after fassert() failure

        1. wiredTiger-beforerecovery.taraa
          140.00 MB
          Jonathan Abrahams
        2. wiredTiger-beforerecovery.tarab
          140.00 MB
          Jonathan Abrahams
        3. wiredTiger-beforerecovery.tarac
          140.00 MB
          Jonathan Abrahams
        4. wiredTiger-beforerecovery.tarag
          31.80 MB
          Jonathan Abrahams
        5. wiredTiger-beforerecovery.tarad
          140.00 MB
          Jonathan Abrahams
        6. wiredTiger-beforerecovery.tarae
          140.00 MB
          Jonathan Abrahams
        7. wiredTiger-beforerecovery.taraf
          140.00 MB
          Jonathan Abrahams
        8. wiredTiger.taraa
          140.00 MB
          Jonathan Abrahams
        9. wiredTiger.tarab
          140.00 MB
          Jonathan Abrahams
        10. wiredTiger.tarae
          140.00 MB
          Jonathan Abrahams
        11. wiredTiger.tarac
          140.00 MB
          Jonathan Abrahams
        12. wiredTiger.tarad
          140.00 MB
          Jonathan Abrahams
        13. wiredTiger.tarag
          29.68 MB
          Jonathan Abrahams
        14. wiredTiger.taraf
          140.00 MB
          Jonathan Abrahams
        15. wiredTiger-afterrecovery.taraa
          140.00 MB
          Jonathan Abrahams
        16. wiredTiger-afterrecovery.tarab
          140.00 MB
          Jonathan Abrahams
        17. wiredTiger-afterrecovery.tarac
          140.00 MB
          Jonathan Abrahams
        18. wiredTiger-afterrecovery.tarad
          140.00 MB
          Jonathan Abrahams
        19. wiredTiger-afterrecovery.tarae
          140.00 MB
          Jonathan Abrahams
        20. wiredTiger-afterrecovery.taraf
          140.00 MB
          Jonathan Abrahams
        21. wiredTiger-afterrecovery.tarag
          29.14 MB
          Jonathan Abrahams
        22. wiredTiger-firstrun.tar
          11 kB
          Jonathan Abrahams
        23. logs.tar
          343 kB
          Jonathan Abrahams
        24. powertest.sh
          32 kB
          Jonathan Abrahams

            Assignee:
            sue.loverso@mongodb.com Susan LoVerso
            Reporter:
            jonathan.abrahams Jonathan Abrahams (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved:
              None
              None
              None
              None