Invariant failure, WT_NOTFOUND: item not found

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Critical - P2
    • None
    • Affects Version/s: 4.4.0, 4.4.1
    • Component/s: Stability, WiredTiger
    • None
    • ALL
    • Hide

      Unfortunately I cannot get it reproduced on will yet.

      Show
      Unfortunately I cannot get it reproduced on will yet.
    • 119
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None

      Hi

       

      We have been getting infrequent (1-2 times a day) aborts of this kind lately. This happened on 4.4.1 and 4.4.0 too. We have had years without issue (3-node replica set), upgrading as new versions come around. It isn't tied to specific times either.

      This is on an AWS I3 instance with nvme drive, formatted to xfs.

      Snippet of the logs here and in the attachement.

      https://pastebin.com/uqauh8H0

      We have tried removing the DB path and resyncing from a secondary, but this did not fix it.

      What could cause this? Could it be a specific query? Could the disk itself be corrupt? How can I help pinpointing the issue?

        1. image-2020-10-15-16-49-06-849.png
          69 kB
          Luke Pearson
        2. MongoDB Abort.json
          22 kB
          Pieter Jordaan
        3. mongo-log.json
          10 kB
          _zbbjornson@gmail.com

            Assignee:
            Jonathan Streets (Inactive)
            Reporter:
            Pieter Jordaan
            Votes:
            1 Vote for this issue
            Watchers:
            30 Start watching this issue

              Created:
              Updated:
              Resolved: