[SERVER-3053] Assertion error on mongod server start Created: 06/May/11  Updated: 30/Mar/12  Resolved: 08/Sep/11

Status: Closed
Project: Core Server
Component/s: Stability
Affects Version/s: 1.6.5, 1.8.0-rc2, 1.8.1
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Maksym Polshcha Assignee: Spencer Brody (Inactive)
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

FreeBSD 7.1-RELEASE-p5 amd64


Operating System: FreeBSD
Participants:

 Description   

I got the following error on every server start using all the affected versions (I've not tried the others).
The error occured for the first time right after installation at the first run. No other versions of mongo installed.
Database is absolutely empty, --repairt option affects the same error.

I had tried:

  • 1.6.5 from ports
  • 1.8.1. and 1.8.0-rc2 from sources

Fri May 6 12:31:18 [initandlisten] MongoDB starting : pid=12866 port=27017 dbpa
th=./mongo 64-bit
Fri May 6 12:31:18 [initandlisten] db version v1.8.2-rc0, pdfile version 4.5
Fri May 6 12:31:18 [initandlisten] git version: nogitversion
Fri May 6 12:31:18 [initandlisten] build sys info: FreeBSD mobile.sterch.net 7.1-RELEASE-p5 FreeBSD 7.1-RELEASE-p5 #3: Thu Oct 28 17:11:46 EEST 2010 root@mobile.sterch.net:/usr/obj/usr/src/sys/MOBILE amd64 BOOST_LIB_VERSION=1_37
Fri May 6 12:31:18 [initandlisten] Assertion: 10334:Invalid BSONObj size: 0 (0x00000000) first element: EOO
Fri May 6 12:31:18 [initandlisten] exception in initAndListen std::exception: Invalid BSONObj size: 0 (0x00000000) first element: EOO, terminating
Fri May 6 12:31:18 dbexit:
Fri May 6 12:31:18 [initandlisten] shutdown: going to close listening sockets...
Fri May 6 12:31:18 [initandlisten] shutdown: going to flush diaglog...
Fri May 6 12:31:18 [initandlisten] shutdown: going to close sockets...
Fri May 6 12:31:18 [initandlisten] shutdown: waiting for fs preallocator...
Fri May 6 12:31:18 [initandlisten] shutdown: closing all files...
Fri May 6 12:31:18 closeAllFiles() finished
Fri May 6 12:31:18 [initandlisten] shutdown: removing fs lock...
Fri May 6 12:31:18 dbexit: really exiting now



 Comments   
Comment by Spencer Brody (Inactive) [ 08/Sep/11 ]

Resolving this for now, though I'll keep my ears open for anyone else seeing the same thing.

Comment by Mathias Stearn [ 04/Aug/11 ]

I think you are getting all of the FreeBSD issues. Let me know if you need any help tracking this down.

Comment by Maksym Polshcha [ 11/May/11 ]

1. I have no FreeBSD8 installed on our servers still. We use 7.1 as production environment and upgrade is impossible.
2. gcc version 4.2.1

Comment by Mathias Stearn [ 10/May/11 ]

Hmm, that shouldn't be happening...

What version of g++ was this compiled with? Could you try compiling on FreeBSD8?

Comment by Maksym Polshcha [ 10/May/11 ]

Starting program: /opt/mongo/bin/mongod --port 27017 --dbpath mongo -vvvvv
[New LWP 100866]
[New Thread 0x801e02180 (LWP 100866)]
[New Thread 0x801e02300 (LWP 101562)]
[New Thread 0x801e02480 (LWP 101564)]
Tue May 10 22:27:27 isInRangeTest passed
Tue May 10 22:27:27 BackgroundJob starting: DataFileSync
Tue May 10 22:27:27 shardKeyTest passed
Tue May 10 22:27:27 shardObjTest passed
Tue May 10 22:27:27 must specify both start and end of balancing window:

{ start : 1 }

Tue May 10 22:27:27 must specify both start and end of balancing window:

{ stop: 1 }

Tue May 10 22:27:27 cannot parse active window (use hh:mm 24hs format):

{ start: "21:30", stop: "28:35" }

Tue May 10 22:27:27 BalancingWidowObjTest passed
Tue May 10 22:27:27 versionCmpTest passed
Tue May 10 22:27:27 [initandlisten] MongoDB starting : pid=32870 port=27017 dbpa
th=mongo 64-bit
Tue May 10 22:27:27 [initandlisten] db version v1.8.2-rc0, pdfile version 4.5
Tue May 10 22:27:27 [initandlisten] git version: nogitversion
Tue May 10 22:27:27 [initandlisten] build sys info: FreeBSD mobile.sterch.net 7.
1-RELEASE-p5 FreeBSD 7.1-RELEASE-p5 #3: Thu Oct 28 17:11:46 EEST 2010 root@m
obile.sterch.net:/usr/obj/usr/src/sys/MOBILE amd64 BOOST_LIB_VERSION=1_37
[New Thread 0x801e02780 (LWP 101572)]
[Switching to Thread 0x801e02180 (LWP 100866)]

Breakpoint 1, mongo::msgasserted (msgid=10334,
msg=0x801e3e938 "Invalid BSONObj size: 0 (0x00000000) first element: EOO")
at util/assert_util.cpp:98
98 void msgasserted(int msgid, const char *msg) {

(gdb) backtrace
#0 mongo::msgasserted (msgid=10334,
msg=0x801e3e938 "Invalid BSONObj size: 0 (0x00000000) first element: EOO")
at util/assert_util.cpp:98
#1 0x00000000004c4704 in mongo::BSONObj::_assertInvalid (this=0x7fffffffce30)
at assert_util.h:139
#2 0x00000000004dbc48 in BSONObj (this=0x7fffffffce30,
msgdata=0x801e3e82d "", ifree=Variable "ifree" is not available.
) at bsonobj.h:409
#3 0x0000000000514527 in mongo::DbMessage::nextJsObj (this=0x7fffffffcf90)
at dbmessage.h:149
#4 0x000000000075ebbf in QueryMessage (this=0x7fffffffcf40,
d=@0x7fffffffcf90) at dbmessage.h:201
#5 0x0000000000751df1 in receivedQuery (c=@0x801e20180,
dbresponse=@0x7fffffffd4b0, m=@0x7fffffffd510) at db/instance.cpp:170
#6 0x000000000075a68a in mongo::assembleResponse (m=@0x7fffffffd510,
dbresponse=@0x7fffffffd4b0, client=@0xacd5a0) at db/instance.cpp:283
#7 0x000000000075bc2b in mongo::DBDirectClient::call (this=Variable "this" is not available.)
at db/instance.cpp:651
#8 0x000000000057ee0e in mongo::DBClientCursor::init (this=0x801e530c0)
at client/dbclientcursor.cpp:53
#9 0x000000000055e5da in mongo::DBClientBase::query (this=Variable "this" is not available.)
at client/dbclient.cpp:611
#10 0x0000000000750346 in mongo::DBDirectClient::query (this=Variable "this" is not available.)
at db/instance.cpp:671
#11 0x00000000008bf4f9 in mongo::clearTmpCollections () at db/db.cpp:404
#12 0x00000000008bfecb in mongo::_initAndListen (listenPort=27017,
appserverLoc=Variable "appserverLoc" is not available.
) at db/db.cpp:517
#13 0x00000000008c1062 in mongo::initAndListen (listenPort=Variable "listenPort"
is not available.
) at db/db.cpp:558
#14 0x00000000008c78a9 in main (argc=6, argv=0x7fffffffe9e8) at db/db.cpp:1059

Comment by Mathias Stearn [ 10/May/11 ]

Do you have gdb installed, or can you install gdb easily? It would be very helpful for debugging this.

If you've never run gdb before, all you need to do is type these lines at the prompt that comes up after running "gdb /path/to/mongod"

break mongo::msgasserted
run --port 27017 --dbpath mongo -vvvvv
<let it run until it returns to the shell>
backtrace

then attach the full output

Comment by Maksym Polshcha [ 06/May/11 ]

The previous error message was put out when 1.6.5 was starting.
The following for 1.8.2-rc0
maxp@mobile:~> /opt/mongo/bin/mongod --port 27017 --dbpath mongo -vvvvv
Fri May 6 21:03:46 isInRangeTest passed
Fri May 6 21:03:46 BackgroundJob starting: DataFileSync
Fri May 6 21:03:46 shardKeyTest passed
Fri May 6 21:03:46 shardObjTest passed
Fri May 6 21:03:46 must specify both start and end of balancing window:

{ start : 1 }

Fri May 6 21:03:46 must specify both start and end of balancing window:

{ stop: 1 }

Fri May 6 21:03:46 cannot parse active window (use hh:mm 24hs format):

{ start: "21:30", stop: "28:35" }

Fri May 6 21:03:46 BalancingWidowObjTest passed
Fri May 6 21:03:46 versionCmpTest passed
Fri May 6 21:03:46 [initandlisten] MongoDB starting : pid=31029 port=27017 dbpa
th=mongo 64-bit
Fri May 6 21:03:46 [initandlisten] db version v1.8.2-rc0, pdfile version 4.5
Fri May 6 21:03:46 [initandlisten] git version: nogitversion
Fri May 6 21:03:46 [initandlisten] build sys info: FreeBSD mobile.sterch.net 7.
1-RELEASE-p5 FreeBSD 7.1-RELEASE-p5 #3: Thu Oct 28 17:11:46 EEST 2010 root@m
obile.sterch.net:/usr/obj/usr/src/sys/MOBILE amd64 BOOST_LIB_VERSION=1_37
Fri May 6 21:03:46 [initandlisten] Assertion: 10334:Invalid BSONObj size: 0 (0x
00000000) first element: EOO
Fri May 6 21:03:46 [initandlisten] exception in initAndListen std::exception: I
nvalid BSONObj size: 0 (0x00000000) first element: EOO, terminating
Fri May 6 21:03:46 dbexit:
Fri May 6 21:03:46 [initandlisten] shutdown: going to close listening sockets..
.
Fri May 6 21:03:46 [initandlisten] shutdown: going to flush diaglog...
Fri May 6 21:03:46 [initandlisten] shutdown: going to close sockets...
Fri May 6 21:03:46 [initandlisten] shutdown: waiting for fs preallocator...
Fri May 6 21:03:46 [initandlisten] shutdown: closing all files...
Fri May 6 21:03:46 closeAllFiles() finished
Fri May 6 21:03:46 [initandlisten] shutdown: removing fs lock...
Fri May 6 21:03:46 dbexit: really exiting now

Comment by Maksym Polshcha [ 06/May/11 ]

Output with -vvvvv. There is an unprintable character in 'first element:?'

maxp@mobile:~> mongod --port 27017 --dbpath mongo -vvvvv
Fri May 6 20:58:58 MongoDB starting : pid=30938 port=27017 dbpath=mongo 64-bit
Fri May 6 20:58:58 db version v1.6.6-pre-, pdfile version 4.5
Fri May 6 20:58:58 git version: nogitversion
Fri May 6 20:58:58 sys info: FreeBSD mobile.sterch.net 7.1-RELEASE-p5 FreeBSD 7.1-RELEASE-p5 #3: Thu Oct 28 17:11:46 EEST 2010 root@mobile.sterch.net:/usr/obj/usr/src/sys/MOBILE amd64 BOOST_LIB_VERSION=1_37
Fri May 6 20:58:58 [initandlisten] Assertion: 10334:Invalid BSONObj spec size:1342177280 (00000050) first element:?: ?type=78
Fri May 6 20:58:58 [initandlisten] exception in initAndListen std::exception: Invalid BSONObj spec size: 1342177280 (00000050) first element:?: ?type=78 , terminating
Fri May 6 20:58:58 dbexit:

Fri May 6 20:58:58 [initandlisten] shutdown: going to close listening sockets..
.
Fri May 6 20:58:58 [initandlisten] shutdown: going to flush oplog...
Fri May 6 20:58:58 [initandlisten] shutdown: going to close sockets...
Fri May 6 20:58:58 [initandlisten] shutdown: waiting for fs preallocator...
Fri May 6 20:58:58 [initandlisten] shutdown: closing all files...
Fri May 6 20:58:58 closeAllFiles() finished

Fri May 6 20:58:58 [initandlisten] shutdown: removing fs lock...
Fri May 6 20:58:58 dbexit: really exiting now

Comment by Eliot Horowitz (Inactive) [ 06/May/11 ]

Its likely a linking issue or something with the binary.
Can you startup with -vvv?
If nothing interesting there, can you start it in gdb and send the trace?

Generated at Thu Feb 08 03:01:56 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.