Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-8245

Fix eviction hang during importCollection

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • 8
    • Storage - Ra 2021-11-01

      This ticket refers to BF-23014, where thread #65 is in the middle of a importCollection operation in MongoDB, and during that high level operation, calls WT_CURSOR::search (via MDB sizeStorer), triggering eviction in the application thread, which stalls.

      Stack trace from the BF:

      Thread 65: "conn38" (Thread 0x7f29e85d1700 (LWP 367))
      [2021/10/17 02:55:05.938] #0  0x00007f2a0ed63878 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
      [2021/10/17 02:55:05.938] #1  0x00005593d2a767e9 in __wt_cond_wait_signal (session=session@entry=0x5593d8eef1f8, cond=0x5593d8eaf680, usecs=usecs@entry=10000, run_func=run_func@entry=0x0, signalled=signalled@entry=0x7f29e85cb480) at src/third_party/wiredtiger/src/os_posix/os_mtx_cond.c:115
      [2021/10/17 02:55:05.938] #2  0x00005593d2a4c77e in __wt_cond_wait (run_func=0x0, usecs=10000, cond=<optimized out>, session=0x5593d8eef1f8) at src/third_party/wiredtiger/src/include/misc_inline.h:19
      [2021/10/17 02:55:05.938] #3  __wt_cache_eviction_worker (session=session@entry=0x5593d8eef1f8, busy=<optimized out>, busy@entry=false, readonly=readonly@entry=false, pct_full=<optimized out>) at src/third_party/wiredtiger/src/evict/evict_lru.c:2432
      [2021/10/17 02:55:05.938] #4  0x00005593d2aeeac4 in __wt_cache_eviction_check (busy=false, didworkp=0x0, didworkp=0x0, readonly=false, busy=false, session=<optimized out>) at src/third_party/wiredtiger/src/include/cache_inline.h:531
      [2021/10/17 02:55:05.938] #5  __cursor_enter (session=<optimized out>) at src/third_party/wiredtiger/src/include/cursor_inline.h:190
      [2021/10/17 02:55:05.938] #6  __wt_cursor_func_init (reenter=true, reenter=true, cbt=0x5593db860c00) at src/third_party/wiredtiger/src/include/cursor_inline.h:436
      [2021/10/17 02:55:05.938] #7  __wt_btcur_search (cbt=cbt@entry=0x5593db860c00) at src/third_party/wiredtiger/src/btree/bt_cursor.c:565
      [2021/10/17 02:55:05.938] #8  0x00005593d2a39508 in __curfile_search (cursor=0x5593db860c00) at src/third_party/wiredtiger/src/cursor/cur_file.c:201
      [2021/10/17 02:55:05.938] #9  0x00005593d2a03480 in mongo::WiredTigerSizeStorer::load (this=0x5593d8e44040, uri=...) at src/mongo/db/storage/wiredtiger/wiredtiger_size_storer.cpp:112
      [2021/10/17 02:55:05.938] #10 0x00005593d29ee26c in mongo::WiredTigerRecordStore::WiredTigerRecordStore (this=0x5593e4b74000, kvEngine=0x5593d8e22b80, ctx=0x5593e2fafa00, params=...) at src/mongo/base/string_data.h:66
      [2021/10/17 02:55:05.938] #11 0x00005593d29ee679 in mongo::StandardWiredTigerRecordStore::StandardWiredTigerRecordStore (this=0x5593e4b74000, kvEngine=0x5593d8e22b80, opCtx=0x5593e2fafa00, params=...) at /opt/mongodbtoolchain/revisions/4ac427ffa2fb12ffce7028023dae1775a06e9bf5/stow/gcc-v3.7vm/include/c++/8.5.0/bits/basic_string.h:940
      [2021/10/17 02:55:05.938] #12 0x00005593d29c8f50 in std::make_unique<mongo::StandardWiredTigerRecordStore, mongo::WiredTigerKVEngine*, mongo::OperationContext*&, mongo::WiredTigerRecordStore::Params&> () at /opt/mongodbtoolchain/revisions/4ac427ffa2fb12ffce7028023dae1775a06e9bf5/stow/gcc-v3.7vm/include/c++/8.5.0/bits/basic_string.h:940
      [2021/10/17 02:55:05.938] #13 mongo::WiredTigerKVEngine::getRecordStore (this=0x5593d8e22b80, opCtx=0x5593e2fafa00, ns=..., ident="collection-1580-4548991179410233755", options=...) at src/mongo/db/storage/wiredtiger/wiredtiger_kv_engine.cpp:1504
      [2021/10/17 02:55:05.938] #14 0x00005593d3096908 in mongo::DurableCatalogImpl::importCollection (this=0x5593d8e447c0, opCtx=0x5593e2fafa00, nss=..., metadata=..., storageMetadata=unowned BSONObj 6341 bytes @ 0x5593e646b4fb - unprintable or invalid, uuidOption=mongo::DurableCatalog::ImportCollectionUUIDOption::kKeepOld) at src/mongo/base/string_data.h:66
      [2021/10/17 02:55:05.938] #15 0x00005593d29a7506 in mongo::<lambda()>::operator()(void) const (__closure=0x7f29e85cc740) at src/mongo/db/modules/enterprise/src/live_import/import_collection.cpp:278
      [2021/10/17 02:55:05.938] #16 0x00005593d29a8811 in mongo::writeConflictRetry<mongo::importCollection(mongo::OperationContext*, const mongo::UUID&, const mongo::NamespaceString&, long long int, long long int, const mongo::BSONObj&, const mongo::BSONObj&, bool)::<lambda()> > (f=..., ns=..., opStr=..., opCtx=0x5593e2fafa00) at /opt/mongodbtoolchain/revisions/4ac427ffa2fb12ffce7028023dae1775a06e9bf5/stow/gcc-v3.7vm/include/c++/8.5.0/bits/unique_ptr.h:345
      [2021/10/17 02:55:05.939] #17 mongo::importCollection (opCtx=<optimized out>, importUUID=..., nss=..., numRecords=<optimized out>, dataSize=<optimized out>, catalogEntry=..., storageMetadata=..., isDryRun=<optimized out>) at src/mongo/db/modules/enterprise/src/live_import/import_collection.cpp:204
      [2021/10/17 02:55:05.939] #18 0x00005593d29a9bba in mongo::runImportCollectionCommand (opCtx=<optimized out>, opCtx@entry=0x5593e2fafa00, collectionProperties=..., force=<optimized out>) at build/cached/mongo/db/modules/enterprise/src/live_import/collection_properties_gen.h:95
      [2021/10/17 02:55:05.939] #19 0x00005593d299e1d4 in mongo::(anonymous namespace)::ImportCollectionCommand::Invocation::typedRun (opCtx=0x5593e2fafa00, this=0x5593e15bacc0) at build/cached/mongo/db/modules/enterprise/src/live_import/commands/import_collection_gen.h:59

            Assignee:
            chenhao.qu@mongodb.com Chenhao Qu
            Reporter:
            donald.anderson@mongodb.com Donald Anderson
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: