Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-98294

Avoid deadlock state when hitting an invariant within the logging code for handling an invariant/tassert

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization

      If we hit an invariant in the logging code while processing an invariant or tassert that happened during normal code flow, we hit some sort of deadlock state. For the first combination (invariant during logging while handling invariant during normal code flow), the sequence of events is that the first invariant calls into invariantFailedWithMsg which calls callAbort which calls std::abort(). We have installed some custom signal handler functions (see here) which get invoked when we process the specified signals. This means that next we call abruptQuitAction which calls abruptQuit, which invokes the logging via printErrorBlock. While logging, we hit an invariant again, but this time the behavior is different when we reach callAbort. Notice that the call to std::abort() is within a function static (static variable declared in a function):

      [[maybe_unused]] static auto initOnce = (std::abort(), 0);

      This means that we initialize the value of initOnce upon the first entry of callAbort and then use that value for any subsequent calls. However, "initialization recursively enters the block in which the variable is being initialized", and according to the cpp docs this leads to undefined behavior. In our case, the thread handling the connection to the shell hangs. I tried attaching gdb to the process and it showed that the thread was waiting for a condition variable (I have a screenshot of this in this doc, image 5).

            Assignee:
            Unassigned Unassigned
            Reporter:
            militsa.sotirova@mongodb.com Militsa Sotirova
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: