Details
-
Question
-
Resolution: Done
-
Major - P3
-
None
-
None
-
None
-
None
Description
Hello,
We use mongo in a Windows container. We use the image 4.2-windowsservercore-1809.
It works in many cases, but we have a customer where mongo crashes after 15-20-30 minutes. I saw the issue 33291 (https://jira.mongodb.org/browse/SERVER-33291), but doesn't seem to be our issue.
In log, we have nothing interesting. We receive valid log I think and we saw a reboot without any crash reason. What happen ? No idea. We ran the container with the env. variable MONGODB_SYSTEM_LOG_VERBOSITY=5, but nothing related to a crash. And after the crash, it can restart since there is an issue with the WiredTiger.lock file.
How could we get more information on the crash in logs ?
Here is a few logs before it restart and the issue that we have after with the WiredTiger.lock file. Thank you for your help.
2020-12-07T15:19:17.786+0100 I NETWORK [conn53] end connection 172.22.36.234:49349 (57 connections now open)
|
2020-12-07T15:19:17.787+0100 I NETWORK [conn54] end connection 172.22.36.234:49350 (56 connections now open)
|
2020-12-07T15:19:18.280+0100 I NETWORK [listener] connection accepted from 172.22.36.234:49355 #59 (57 connections now open)
|
2020-12-07T15:29:09.578+0100 I NETWORK [conn1] end connection 172.22.38.187:49413 (56 connections now open)
|
2020-12-07T15:29:09.582+0100 I NETWORK [listener] connection accepted from 172.22.38.187:49584 #60 (57 connections now open)
|
2020-12-07T15:29:09.583+0100 I NETWORK [conn60] received client metadata from 172.22.38.187:49584 conn60: \{ driver: { name: "mongo-rust-driver", version: "1.1.1" }, os: \{ type: "windows", architecture: "x86_64", version: "Windows Server 2016 10.0.17763" }
|
2020-12-07T15:34:29.713+0100 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
|
2020-12-07T15:34:30.470+0100 W ASIO [main] No TransportLayer configured during NetworkInterface startup
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] MongoDB starting : pid=9740 port=27017 dbpath=C:\data\db\ 64-bit host=42eb81c3b9db
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] db version v4.2.11
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] git version: ea38428f0c6742c7c2c7f677e73d79e17a2aab96
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] allocator: tcmalloc
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] modules: none
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] build environment:
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] distmod: 2012plus
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] distarch: x86_64
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] target_arch: x86_64
|
2020-12-07T15:34:30.632+0100 I CONTROL [initandlisten] options: \{ net: { bindIp: "*" } }
|
2020-12-07T15:34:30.753+0100 W STORAGE [initandlisten] Detected unclean shutdown - C:\data\db\mongod.lock is not empty.
|
2020-12-07T15:34:30.753+0100 I STORAGE [initandlisten] Detected data files in C:\data\db\ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
|
2020-12-07T15:34:30.753+0100 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
|
2020-12-07T15:34:30.753+0100 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=3583M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
|
2020-12-07T15:34:30.755+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device
|
2020-12-07T15:34:30.755+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device
|
2020-12-07T15:34:30.755+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device
|
2020-12-07T15:34:30.755+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device
|
2020-12-07T15:34:30.756+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device
|
2020-12-07T15:34:30.756+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device Raw: [1607351670:754772][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device
|
2020-12-07T15:34:30.756+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:756443][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device Raw: [1607351670:756443][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device
|
2020-12-07T15:34:30.840+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:839774][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device Raw: [1607351670:839774][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device
|
2020-12-07T15:34:30.840+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:840774][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device Raw: [1607351670:840774][9740:140717493998176], wiredtiger_open: __win_file_lock, 241: C:\data\db\\WiredTiger.lock: handle-lock: LockFile: The process cannot access the file because another process has locked a portion of the file.\r\n: Resource device
|
2020-12-07T15:34:30.840+0100 E STORAGE [initandlisten] WiredTiger error (16) [1607351670:840774][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device Raw: [1607351670:840774][9740:140717493998176], wiredtiger_open: __conn_single, 1708: WiredTiger database is already being managed by another process: Resource device
|
2020-12-07T15:34:30.840+0100 W STORAGE [initandlisten] Failed to start up WiredTiger under any compatibility version.
|
2020-12-07T15:34:30.840+0100 F STORAGE [initandlisten] Reason: 16: Resource device
|
2020-12-07T15:34:30.841+0100 F - [initandlisten] Fatal Assertion 28595 at src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp 925
|
2020-12-07T15:34:30.841+0100 F - [initandlisten] \n\n***aborting after fassert() failure\n\n
|
2020-12-07T15:34:40.278+0100 I CONTROL [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
|