Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-56222

Startup logging should provide regular status updates for the undergoing recovery

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Logging
    • Labels:
    • Storage Engines

      It has been observed in the most recent versions of MongoDB (4.4.5) that startup recovery of an instance with a large data set can take several hours.

      This is problematic for TSEs and for the users as the status of the mongod process is not entirely clear.

      For example:

      • There is no indication how many phases or steps the recovery process has in total
      • Where the current progress is
      • How many steps are outstanding

      To illustrate, it's been observed that a mongod spends multiple hours recovering after having a message similar to the following printed in the log with no other clear signs of progress:

      {"t":{"$date":"2021-04-21T00:51:14.703+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1618966274:703153][5978:0x7f67cef96bc0], file:collection-2-1893086824266225355.wt, txn-recover: [WT_VERB_RECOVERY_PROGRESS] Recovering log 62628 through 62628"}}
      {"t":{"$date":"2021-04-21T00:51:14.759+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1618966274:759611][5978:0x7f67cef96bc0], file:collection-2-1893086824266225355.wt, txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global recovery timestamp: (1618965463, 1)"}}
      {"t":{"$date":"2021-04-21T00:51:14.759+00:00"},"s":"I",  "c":"STORAGE",  "id":22430,   "ctx":"initandlisten","msg":"WiredTiger message","attr":{"message":"[1618966274:759674][5978:0x7f67cef96bc0], file:collection-2-1893086824266225355.wt, txn-recover: [WT_VERB_RECOVERY | WT_VERB_RECOVERY_PROGRESS] Set global oldest timestamp: (1618965458, 1)"}}

            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            dmitry.ryabtsev@mongodb.com Dmitry Ryabtsev
            9 Vote for this issue
            15 Start watching this issue