Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24407

too many WT Cache Eviction cause server hang

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 3.2.6
    • Component/s: WiredTiger
    • Labels:
      None
    • ALL
    • Hide

      every document size is about 500B.
      70+ mongoimport processes concurrent working.

      wait wt cache full, and happen evcition.

      mongostat watch the commands, and pstack mongod

      Show
      every document size is about 500B. 70+ mongoimport processes concurrent working. wait wt cache full, and happen evcition. mongostat watch the commands, and pstack mongod

      A lot of process insert into MongodDB，when wt cache full，will trigger eviction，even in MongoDB insert thread.

      But evict is too slow, and so many race conditions, CPU Heavy, Server Hang, can't server any request.

       131 Thread 145 (Thread 0x7fb86d464700 (LWP 31157)):
 132 #0  0x0000000001aa35bb in __wt_txn_update_oldest ()
 133 #1  0x0000000001a49a8d in __wt_evict ()
 134 #2  0x0000000001a46a9f in __evict_page ()
 135 #3  0x0000000001a490f6 in __wt_cache_eviction_worker ()
 136 #4  0x0000000001a963b8 in __session_begin_transaction ()
 137 #5  0x00000000010a513c in mongo::WiredTigerRecoveryUnit::_txnOpen(mongo::OperationContext*) ()
 138 #6  0x00000000010a52f7 in mongo::WiredTigerRecoveryUnit::getSession(mongo::OperationContext*) ()
 139 #7  0x00000000010a53ca in mongo::WiredTigerCursor::WiredTigerCursor(std::basic_string<char, std::char_traits<char>, std::al     locator<char> > const&, unsigned long, bool, mongo::OperationContext*) ()
 140 #8  0x000000000109aa11 in mongo::WiredTigerRecordStore::insertRecords(mongo::OperationContext*, std::vector<mongo::Record,      std::allocator<mongo::Record> >*, bool) ()

        1. diagnostic1.png
          diagnostic1.png
          363 kB
        2. mongo.log.tgz
          4.81 MB
        3. pstack.log
          312 kB
        4. screenshot-1.png
          screenshot-1.png
          2.03 MB
        5. top.log
          15 kB

            Assignee:
            Unassigned Unassigned
            Reporter:
            ych.tiger@gmail.com YANG Chenghu
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: