[SERVER-24155] can't start mongod: aborting after invariant() failure Created: 16/May/16  Updated: 22/Apr/19  Resolved: 16/May/16

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: 3.2.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: alfredopacino Assignee: Kelsey Schubert
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

Hi, I get this error trying to start my mongo deamon:

NOTE I don't see any sizeStorer.wt in my DB folder.

2016-05-16T17:23:06.521+0200 I CONTROL [main] Hotfix KB2731284 or later update is not installed, will zero-out data files
2016-05-16T17:23:06.525+0200 I CONTROL [initandlisten] MongoDB starting : pid=7092 port=27017 dbpath=C:\charterclick\data 64-bit host=admin-PC
2016-05-16T17:23:06.525+0200 I CONTROL [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2016-05-16T17:23:06.525+0200 I CONTROL [initandlisten] db version v3.2.6
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] git version: 05552b562c7a0b3143a729aaa0838e558dc49b25
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1p-fips 9 Jul 2015
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] allocator: tcmalloc
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] modules: none
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] build environment:
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] distmod: 2008plus-ssl
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] distarch: x86_64
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] target_arch: x86_64
2016-05-16T17:23:06.526+0200 I CONTROL [initandlisten] options: { storage:

{ dbPath: "C:\charterclick\data" }

}
2016-05-16T17:23:06.528+0200 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60
,log_size=2GB),statistics_log=(wait=0),
2016-05-16T17:23:07.417+0200 E STORAGE [initandlisten] WiredTiger (-28998) [1463412187:416287][7092:2000109280], file:sizeStorer.wt, WT_SESSION.open_cursor: C:\charterclick\data\sizeStorer.wt: handle-open: CreateFileA: Impossibile trovare il file specificato.
2016-05-16T17:23:07.417+0200 I - [initandlisten] Invariant failure: ret resulted in status UnknownError: -28998: Impossibile trovare il file specificato.
at src\mongo\db\storage\wiredtiger\wiredtiger_size_storer.cpp 67
2016-05-16T17:23:09.234+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\util\stacktrace_windows.cpp(174) mongo::printStackTrace+0x43
2016-05-16T17:23:09.234+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\util\log.cpp(136) mongo::logContext+0xb0
2016-05-16T17:23:09.234+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\util\assert_util.cpp(164) mongo::invariantOKFailed+0x139
2016-05-16T17:23:09.235+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_size_storer.cpp(67) mongo::WiredTigerSizeStorer::WiredTigerSizeStorer+0x1d1
2016-05-16T17:23:09.235+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp(263) mongo::WiredTigerKVEngine::WiredTigerKVEngine+0x95a
2016-05-16T17:23:09.235+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\storage\wiredtiger\wiredtiger_init.cpp(85) mongo::`anonymous namespace'::WiredTigerFactory::create+0x1a3
2016-05-16T17:23:09.235+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\service_context_d.cpp(148) mongo::ServiceContextMongoD::initializeGlobalStorageEngine+0x88a
2016-05-16T17:23:09.235+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\db.cpp(560) mongo::_initAndListen+0x5b6
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\db.cpp(741) mongo::initAndListen+0x27
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\db.cpp(974) mongoDbMain+0x216
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten] mongod.exe ...\src\mongo\db\db.cpp(780) wmain+0x35
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten] mongod.exe f:\dd\vctools\crt\crtw32\dllstuff\crtexe.c(623) __tmainCRTStartup+0x10f
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten] kernel32.dll BaseThreadInitThunk+0xd
2016-05-16T17:23:09.236+0200 I CONTROL [initandlisten]
2016-05-16T17:23:09.236+0200 I - [initandlisten]

***aborting after invariant() failure



 Comments   
Comment by Michael Brenden [ 22/Apr/19 ]

Check the obvious – file limits

ulimit -n

Default file limit of 1,024 is probably nowhere near enough.

Comment by Kelsey Schubert [ 16/May/16 ]

Hi alfredopacino,

Thanks for the quick response. Journaling is enabled by default.

If your $dbpath was modified by git operations, this would most likely explain the behavior you are observing. Please note that data files in the $dbpath should only be committed when the mongod process is shut down or after executing db.fsyncLock(), otherwise the files committed may be in an inconsistent state.

Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-users group or Stack Overflow with the mongodb tag. Questions about using version control on your $dbpath would involve more discussion would be best posted on the mongodb-users group.

Kind regards,
Thomas

Comment by alfredopacino [ 16/May/16 ]

First of all I'm relatively new to mongodb
1. I'm working in local. So no replica set.
2. I can't answer on that. If the default behaviour is jurnaled I am (is this the default behaviour?).
3. The storage engine is the default one. WiredTiger I guess. The db is on my ssd (the operating system physical device). NO Raid. The OS is Win 7 x64 ultimate.
4. no.
5. no.
6. no backup except just backup of the whole project directory (the db directory is IN the project directory) but I never used backup, so I never overwrited the db files.

Today (before the I noticed the mongod error) I worked on git (removing and add directories and files from my repos, so I operated on the db directory c:\myproject\data through git). Nothing more. I don't think it's related to this issue... although it's the only thing I've done.

Comment by Kelsey Schubert [ 16/May/16 ]

Hi alfredopacino,

Thanks for reporting this issue. To get a better understanding of what is going on, please answer the following questions:

  1. Is this node part of a replica set? If so, are other members affected?
  2. Are you using journaling?
  3. What kind of underlying storage mechanism are you using? Are the storage devices attached locally or over the network? Are the disks SSDs or HDDs? What kind of RAID and/or volume management system are you using?
  4. Have you manipulated (copied or moved) the underlying database files? If so, was mongod running?
  5. Have you ever restored this instance from backups?
  6. What method do you use to create backups?

Thank you,
Thomas

Generated at Thu Feb 08 04:05:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.