Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21078

Segfault from race between getlasterror with fsync:true and clean database shutdown

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Minor - P4 Minor - P4
    • 3.2.0-rc4
    • Affects Version/s: 3.2.0-rc0
    • Component/s: WiredTiger
    • None
    • Fully Compatible
    • ALL
    • Hide

      Run attached bash shell script, fsyncwt.sh, from a directory that contains "mongo" and "mongod" binaries.

      It does the following:

      1. launches mongod
      2. starts a background job that runs getlasterror with fsync:true in a loop
      3. shuts down the mongod using the shutdown command
      4. expects mongod to exit with code 0

      The actual behavior is that eventually (rarely in more than 5 runs for me) the mongod segfaults.

      Show
      Run attached bash shell script, fsyncwt.sh, from a directory that contains "mongo" and "mongod" binaries. It does the following: launches mongod starts a background job that runs getlasterror with fsync:true in a loop shuts down the mongod using the shutdown command expects mongod to exit with code 0 The actual behavior is that eventually (rarely in more than 5 runs for me) the mongod segfaults.
    • QuInt C (11/23/15)

      There appears to be a race between shutting down a mongod instance and running getlasterror with fsync:true when the storage engine is wired tiger and journaling is disabled. When the race goes poorly, mongod segfaults.

      The stack trace via addr2line -i is as follows, with a build of mongod at commit 737bb20fcb9176eb5f664bd874cdaece779d4012.

      /data/rbuild/mongo/src/mongo/util/stacktrace_posix.cpp:171
/data/rbuild/mongo/src/mongo/util/signal_handlers_synchronous.cpp:179
/data/rbuild/mongo/src/mongo/util/signal_handlers_synchronous.cpp:274
??:0
/data/rbuild/mongo/src/third_party/wiredtiger/src/txn/txn_ckpt.c:322
/data/rbuild/mongo/src/third_party/wiredtiger/src/txn/txn_ckpt.c:529 (discriminator 1)
/data/rbuild/mongo/src/third_party/wiredtiger/src/session/session_api.c:1086 (discriminator 3)
/data/rbuild/mongo/src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:299
/data/rbuild/mongo/src/mongo/db/storage/kv/kv_storage_engine.cpp:247
/data/rbuild/mongo/src/mongo/db/write_concern.cpp:244
/data/rbuild/mongo/src/mongo/db/commands/get_last_error.cpp:274 (discriminator 1)
/data/rbuild/mongo/src/mongo/db/dbcommands.cpp:1385
/data/rbuild/mongo/src/mongo/db/dbcommands.cpp:1290
/data/rbuild/mongo/src/mongo/db/commands.cpp:496
/data/rbuild/mongo/src/mongo/db/instance.cpp:293
/data/rbuild/mongo/src/mongo/db/instance.cpp:526
/data/rbuild/mongo/src/mongo/db/db.cpp:170 (discriminator 1)
/data/rbuild/mongo/src/mongo/util/net/message_server_port.cpp:229

        1. fsyncwt.sh
          0.8 kB
          Andy Schwerin

            Assignee:
            geert.bosch@mongodb.com Geert Bosch
            Reporter:
            schwerin@mongodb.com Andy Schwerin
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: