-
Type:
Task
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.2
-
Component/s: WiredTiger
-
Labels:None
-
Storage 2018-02-12
-
(copied to CRM)
Importing to a new collection on a standalone database stored on a NTFS disk results in a WT library Panic. Restarting Mongodb after the panic also results in a panic. Both logs are below
Mongo: 3.6.2 Community
OS: Windows 10 Pro x64
Disk: NTFS with compression flag true
Processor: AMD FX-6300 6 Cores
Memory: 32gb DDR4
Database: WiredTiger
Sharding: No
Replication: No
Additional ServerConfig: maxIndexBuildMemoryUsageMegabytes: 30000
Pre import Collection Index: id (unique)
Import this jsonL Dataset: (59gb file) http://files.pushshift.io/reddit/comments/xz/RC_2015.xz
Import Error Log Section:
Unable to find source-code formatter for language: text. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
2018-01-19T21:15:48.543-0500 I COMMAND [conn20] command reddit.fifteen_sixth_try command: insert { insert: "fifteen_sixth_try", ordered: false, $db: "reddit" } ninserted:1000 keysInserted:2000 numYields:0 reslen:29 locks:{ Global: { acquireCount: { r: 16, w: 16 } }, Database: { acquireCount: { w: 16 } }, Collection: { acquireCount: { w: 16 } } } protocol:op_msg 872ms
2018-01-19T21:15:49.278-0500 E STORAGE [WTCheckpointThread] WiredTiger error (-31802) [1516414549:262111][13116:140726555009360], file:collection-0--4941774138393653127.wt, WT_SESSION.checkpoint: E:\gdrive\datalake\databases\mongodb\collection-0--4941774138393653127.wt: handle-write: WriteFile: failed to write 8192 bytes at offset 187563442176: The requested operation could not be completed due to a file system limitation
: WT_ERROR: non-specific WiredTiger error
2018-01-19T21:15:49.278-0500 E STORAGE [WTCheckpointThread] WiredTiger error (22) [1516414549:278711][13116:140726555009360], file:index-1--4941774138393653127.wt, WT_SESSION.checkpoint: index-1--4941774138393653127.wt: the checkpoint failed, the system must restart: Invalid argument
2018-01-19T21:15:49.279-0500 E STORAGE [WTCheckpointThread] WiredTiger error (-31804) [1516414549:278711][13116:140726555009360], file:index-1--4941774138393653127.wt, WT_SESSION.checkpoint: the process must exit and restart: WT_PANIC: WiredTiger library panic
2018-01-19T21:15:49.280-0500 F - [WTCheckpointThread] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361
2018-01-19T21:15:49.280-0500 F - [WTCheckpointThread]
***aborting after fassert() failure
2018-01-19T21:15:49.524-0500 E STORAGE [conn20] WiredTiger error (-31804) [1516414549:523854][13116:140726555009360], WT_SESSION.commit_transaction: journal/WiredTigerLog.0000013585: fatal log failure: WT_PANIC: WiredTiger library panic
2018-01-19T21:15:49.524-0500 E STORAGE [thread46] WiredTiger error (-31804) [1516414549:523854][13116:140726555009360], log-server: log server error: WT_PANIC: WiredTiger library panic
2018-01-19T21:15:49.524-0500 F - [thread46] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361
2018-01-19T21:15:49.525-0500 F - [thread46]
***aborting after fassert() failure
2018-01-19T21:15:49.524-0500 F - [conn29] Fatal Assertion 28559 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 64
2018-01-19T21:15:49.525-0500 F - [conn29]
***aborting after fassert() failure
2018-01-19T21:15:49.524-0500 F - [conn20] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361
2018-01-19T21:15:49.525-0500 F - [conn20]
***aborting after fassert() failure
And here are the restart logs:
Unable to find source-code formatter for language: text. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
2018-01-20T15:50:10.762-0500 W STORAGE [initandlisten] Recovering data from the last clean checkpoint. 2018-01-20T15:50:10.762-0500 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15821M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress), 2018-01-20T15:50:19.255-0500 I STORAGE [initandlisten] WiredTiger message [1516481419:255018][3060:140726555009360], txn-recover: Main recovery loop: starting at 13579/91027712 2018-01-20T15:50:19.261-0500 I STORAGE [initandlisten] WiredTiger message [1516481419:261018][3060:140726555009360], txn-recover: Recovering log 13579 through 13587 2018-01-20T15:50:20.472-0500 I STORAGE [initandlisten] WiredTiger message [1516481420:472091][3060:140726555009360], file:index-2--4941774138393653127.wt, txn-recover: Recovering log 13580 through 13587 2018-01-20T15:50:25.112-0500 I STORAGE [initandlisten] WiredTiger message [1516481425:111360][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13581 through 13587 2018-01-20T15:50:29.596-0500 I STORAGE [initandlisten] WiredTiger message [1516481429:595623][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13582 through 13587 2018-01-20T15:50:34.473-0500 I STORAGE [initandlisten] WiredTiger message [1516481434:472906][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13583 through 13587 2018-01-20T15:50:39.743-0500 I STORAGE [initandlisten] WiredTiger message [1516481439:743213][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13584 through 13587 2018-01-20T15:50:44.477-0500 I STORAGE [initandlisten] WiredTiger message [1516481444:476490][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13585 through 13587 2018-01-20T15:50:47.681-0500 I STORAGE [initandlisten] WiredTiger message [1516481447:680677][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13586 through 13587 2018-01-20T15:50:48.275-0500 I STORAGE [initandlisten] WiredTiger message [1516481448:275712][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovering log 13587 through 13587 2018-01-20T15:50:50.895-0500 E STORAGE [thread1] WiredTiger error (-31802) [1516481450:894865][3060:140726555009360], file:collection-0--4941774138393653127.wt, eviction-server: E:\gdrive\datalake\databases\mongodb\collection-0--4941774138393653127.wt: handle-write: WriteFile: failed to write 16384 bytes at offset 187592658944: The requested operation could not be completed due to a file system limitation : WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.895-0500 E STORAGE [thread1] WiredTiger error (-31802) [1516481450:894865][3060:140726555009360], eviction-server: cache eviction thread error: WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.895-0500 E STORAGE [thread1] WiredTiger error (-31804) [1516481450:894865][3060:140726555009360], eviction-server: the process must exit and restart: WT_PANIC: WiredTiger library panic 2018-01-20T15:50:50.895-0500 F - [thread1] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361 2018-01-20T15:50:50.895-0500 F - [thread1] ***aborting after fassert() failure 2018-01-20T15:50:50.897-0500 E STORAGE [thread2] WiredTiger error (-31802) [1516481450:896866][3060:140726555009360], file:collection-0--4941774138393653127.wt, eviction-server: E:\gdrive\datalake\databases\mongodb\collection-0--4941774138393653127.wt: handle-write: WriteFile: failed to write 16384 bytes at offset 187592675328: The requested operation could not be completed due to a file system limitation : WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.897-0500 E STORAGE [thread2] WiredTiger error (-31802) [1516481450:897865][3060:140726555009360], eviction-server: cache eviction thread error: WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.901-0500 E STORAGE [thread3] WiredTiger error (-31802) [1516481450:900865][3060:140726555009360], file:collection-0--4941774138393653127.wt, eviction-server: E:\gdrive\datalake\databases\mongodb\collection-0--4941774138393653127.wt: handle-write: WriteFile: failed to write 16384 bytes at offset 187592691712: The requested operation could not be completed due to a file system limitation : WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.901-0500 E STORAGE [thread3] WiredTiger error (-31802) [1516481450:900865][3060:140726555009360], eviction-server: cache eviction thread error: WT_ERROR: non-specific WiredTiger error 2018-01-20T15:50:50.901-0500 E STORAGE [initandlisten] WiredTiger error (-31804) [1516481450:901865][3060:140726555009360], file:sizeStorer.wt, txn-recover: Recovery failed: WT_PANIC: WiredTiger library panic 2018-01-20T15:50:50.902-0500 F - [initandlisten] Fatal Assertion 28558 at src\mongo\db\storage\wiredtiger\wiredtiger_util.cpp 361 2018-01-20T15:50:50.902-0500 F - [initandlisten] ***aborting after fassert() failure
