-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 2.2.1
-
Component/s: Stability
-
None
-
ALL
-
None
-
3
-
None
-
None
-
None
-
None
-
None
-
None
I use mongo replicaset but suddenly it crashed. when attempt to start up with command "./mongod --dbpath=../data --replSet replset_BRG --port 28010 --logpath=../log/log --logappend --fork --directoryperdb", error occurs as follow:
***** SERVER RESTARTED ***** Fri May 9 16:29:45 [initandlisten] MongoDB starting : pid=25550 port=28010 dbpath=/home/work/mongo-server/bin/../data 64-bit host= Fri May 9 16:29:45 [initandlisten] db version v2.2.1, pdfile version 4.5 Fri May 9 16:29:45 [initandlisten] git version: d6764bf8dfe0685521b8bc7b98fd1fab8cfeb5ae Fri May 9 16:29:45 [initandlisten] build info: Linux domU-12-31-39-16-30-A2 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:34:28 EST 2008 x86_64 BOOST_LIB_VERSION=1_49 Fri May 9 16:29:45 [initandlisten] options: { dbpath: "../data", directoryperdb: true, fork: true, logappend: true, logpath: "../log/log" , port: 28010, replSet: "replset_BRG" } Fri May 9 16:29:45 [initandlisten] journal dir=/home/work/mongo-server/bin/../data/journal Fri May 9 16:29:45 [initandlisten] recover begin Fri May 9 16:29:45 [initandlisten] recover lsn: 0 Fri May 9 16:29:45 [initandlisten] recover /home/work/mongo-server/bin/../data/journal/j._0 Fri May 9 16:29:45 [initandlisten] recover cleaning up Fri May 9 16:29:45 [initandlisten] removeJournalFiles Fri May 9 16:29:45 [initandlisten] recover done Fri May 9 16:29:45 [initandlisten] waiting for connections on port 28010 Fri May 9 16:29:45 [websvr] admin web console waiting for connections on port 29010 Fri May 9 16:29:45 [initandlisten] connection accepted from 10.46.23.36:23864 #1 (1 connection now open) Fri May 9 16:29:45 [rsStart] replSet I am *:28010 Fri May 9 16:29:45 [rsStart] replSet STARTUP2 Fri May 9 16:29:45 [rsHealthPoll] replSet member*:28010 is up Fri May 9 16:29:45 [rsHealthPoll] replSet member *:28010 is now in state PRIMARY Fri May 9 16:29:45 [rsHealthPoll] replSet member *:28010 is up Fri May 9 16:29:45 [rsHealthPoll] replSet member *:28010 is now in state ARBITER Fri May 9 16:29:46 [rsSync] replSet still syncing, not yet to minValid optime 536b0dc8:1 Fri May 9 16:29:47 [initandlisten] connection accepted from 10.42.152.12:56039 #2 (2 connections now open) Fri May 9 16:29:47 [conn2] end connection *:56039 (1 connection now open) Fri May 9 16:29:47 [initandlisten] connection accepted from 10.42.152.12:56040 #3 (2 connections now open) Fri May 9 16:29:47 [initandlisten] connection accepted from 10.23.240.150:59369 #4 (3 connections now open) Fri May 9 16:29:47 [initandlisten] connection accepted from 10.23.240.150:59374 #5 (3 connections now open) Fri May 9 16:29:51 [rsBackgroundSync] replSet syncing to: 10.42.152.12:28010 Fri May 9 16:29:51 [rsSync] replSet still syncing, not yet to minValid optime 536b0dc8:1 Fri May 9 16:29:53 [conn5] end connection *:59374 (2 connections now open) Fri May 9 16:29:53 [initandlisten] connection accepted from 10.23.240.150:21190 #6 (3 connections now open) Fri May 9 16:29:56 [rsSyncNotifier] replset setting oplog notifier to 9.42.152.12:28010 Fri May 9 16:29:56 [repl prefetch worker] ERROR: mmap() failed for /home/work/mongo-server/bin/../data/bridge/bridge.59 len:2146435072 errno:12 Cannot allocate memory Fri May 9 16:29:56 [repl prefetch worker] ERROR: mmap failed with out of memory. (64 bit build) Fri May 9 16:29:56 [repl writer worker 1] ERROR: mmap() failed for /home/work/mongo-server/bin/../data/bridge/bridge.59 len:2146435072 errno:12 Cannot allocate memory Fri May 9 16:29:56 [repl writer worker 1] ERROR: mmap failed with out of memory. (64 bit build) Fri May 9 16:29:56 [repl writer worker 1] ERROR: writer worker caught exception: can't map file memory on: { ts: Timestamp 1399524808000| 1, h: 1490830305013686414, v: 2, op: "i", ns: "bridge.track", o: { _id: ObjectId('536b0dc8e662b15490342d01'), bid: 515059940390023161, sit e: 2758320, type: 0, tm: 1399523870, date: 20140508, bdid: "BD1BCA60A411EBF3126DAAFFA7450E0B:FG=1", wt: 6, ref: "http://.com /brgview/?p=010b5fb9f4c42c2ea54a04de2716c98e52c8260578b2ff7d6ce067637d4b7aeef66afd58f0b9a1d5b91f2243d7e8ed138f64097a5166140bc...", url: "h ttp://www.55599120.com/", title: "[1m~S~H灏~T婊ㄦ~@х~W~E[1m~L婚~Y[1m~S~H灏~T婊ㄦ不[1m~V~W[1m~@х~W~E[1m~\~@濂界~Z~D[1m~L婚~Y榛~Q榫~Y姹~_[1m~\~A[1m~E[33;1H~I娑~H[1m~X叉~@婚~X~_[1m~L婚~Y~@~P[1m~E[1m~K[1m~T茬骇[1m~L婚~Y~@~Q" } } Fri May 9 16:29:56 [repl writer worker 1] Fatal Assertion 16360 0x9a28c1 0x967ea3 0x845e7e 0x97614d 0x9e82f9 0xacba3d 0xb91039 0x9a28c1 0x967ea3 0x845e7e 0x97614d 0x9e82f9 0xacba3d 0xb91039 [0x9a28c1] [0x967ea3] [0x845e7e] [0x97614d] [0x9e82f9] [0xacba3d] [0xb91039] Fri May 9 16:29:56 [repl writer worker 1] ***aborting after fassert() failure Fri May 9 16:29:56 Got signal: 6 (Aborted). Fri May 9 16:29:56 Backtrace: 0x9a28c1 0x400c69 0xacfc00 0xacfbcd 0xb624a0 0x967ede 0x845e7e 0x97614d 0x9e82f9 0xacba3d 0xb91039 [0x9a28c1] [0x400c69] [0xacfc00] [0xacfbcd] [0xb624a0] [0x967ede] [0x845e7e] [0x97614d] [0x9e82f9] [0xacba3d] [0xb91039]
memory and disk space is enough:
free -m total used free shared buffers cached Mem: 64350 64322 27 0 98 56113 -/+ buffers/cache: 8110 56240 Swap: 996 974 22
df -hl Filesystem Size Used Avail Use% Mounted on /dev/sda2 8.7G 4.3G 4.5G 49% / /dev/sda3 1.9T 975G 938G 51% /home