Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-13410

split does not install metadata under the dblock

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical - P2
    • Resolution: Fixed
    • 2.6.0-rc2
    • 2.6.0-rc3
    • None
    • None
    • ALL

    Description

      We have observed 3 separate instances of the mongod process dying unexpectedly with no message in the log file. In each instance, the last message in the log file was about to log metadata event.

      In each episodes the mongod in question has been a member of a shard in a cluster. The episodes were observed on three separate physical servers.

      Log files attached.

      In the third episode dmesg said the following:

      INFO: task mongod:29664 blocked for more than 120 seconds.
       
      "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
       
      mongod        D 0000000000000001     0 29664  29663 0x00000080
       
       ffff88066032bdf8 0000000000000086 ffff88066032bdc0 ffff88066032bdbc
       
       ffff88066032bd88 ffff88063fc24500 ffff880028215f80 0000000000000400
       
       ffff88008217fab8 ffff88066032bfd8 000000000000f4e8 ffff88008217fab8
       
      Call Trace:
       
       [<ffffffffa00745c5>] jbd2_log_wait_commit+0xc5/0x140 [jbd2]
       
       [<ffffffff81090d30>] ? autoremove_wake_function+0x0/0x40
       
       [<ffffffffa0074676>] ? __jbd2_log_start_commit+0x36/0x40 [jbd2]
       
       [<ffffffffa009409c>] ext4_sync_file+0x13c/0x250 [ext4]
       
       [<ffffffff811a57a1>] vfs_fsync_range+0xa1/0xe0
       
       [<ffffffff811a584d>] vfs_fsync+0x1d/0x20
       
       [<ffffffff811a588e>] do_fsync+0x3e/0x60
       
       [<ffffffff811a58e0>] sys_fsync+0x10/0x20
       
       [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b

      Attachments

        1. cbi1_mmsrs3_2014_03_29.log
          7 kB
          Cailin Nelson
        2. cbi2_mmsrs2_2014_03_30.log
          6 kB
          Cailin Nelson
        3. cbi8_mmsrs4_2014_03_27.log
          54 kB
          Cailin Nelson

        Issue Links

          Activity

            People

              greg_10gen Greg Studer
              cailin.nelson@mongodb.com Cailin Nelson
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: