[SERVER-10360] cause of 'assertion src/mongo/db/database.cpp:300' Created: 27/Jul/13  Updated: 29/Jul/13  Resolved: 29/Jul/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.4.2
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Ravi Shakya Assignee: Tad Marshall
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Ubuntu 10.04 on Intel x84 core 2 duo


Attachments: PNG File assert_error.png     File mlog    
Participants:

 Description   

While trying to query on some collections on a mongodb database, I keep on getting
error:

{ "$err" : "assertion src/mongo/db/database.cpp:300" }

(Please refer to the attached screenshot assert_error.png)
Could you please inform what could be the cause of this issue? Thanks



 Comments   
Comment by Ravi Shakya [ 29/Jul/13 ]

Hi Tad and team,
Thanks for the information/response. I will purge some data to get it working.
You could resolve/close this issue.

Comment by Tad Marshall [ 28/Jul/13 ]

Since the log is very small, here it is inline:

Sun Jul 28 09:07:26.694 [initandlisten] MongoDB starting : pid=4618 port=27027 dbpath=/home/ravi/standalone 32-bit host=ravi-laptop
Sun Jul 28 09:07:26.694 [initandlisten] 
Sun Jul 28 09:07:26.694 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
Sun Jul 28 09:07:26.694 [initandlisten] **       32 bit builds are limited to less than 2GB of data (or less with --journal).
Sun Jul 28 09:07:26.694 [initandlisten] **       Note that journaling defaults to off for 32 bit and is currently off.
Sun Jul 28 09:07:26.694 [initandlisten] **       See http://dochub.mongodb.org/core/32bit
Sun Jul 28 09:07:26.694 [initandlisten] 
Sun Jul 28 09:07:26.694 [initandlisten] db version v2.4.3
Sun Jul 28 09:07:26.694 [initandlisten] git version: fe1743177a5ea03e91e0052fb5e2cb2945f6d95f
Sun Jul 28 09:07:26.694 [initandlisten] build info: Linux bs-linux32.10gen.cc 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_49
Sun Jul 28 09:07:26.694 [initandlisten] allocator: system
Sun Jul 28 09:07:26.694 [initandlisten] options: { dbpath: "/home/ravi/standalone", logpath: "/home/ravi/mongologs/mlog", port: 27027 }
Sun Jul 28 09:07:26.797 [websvr] admin web console waiting for connections on port 28027
Sun Jul 28 09:07:26.797 [initandlisten] waiting for connections on port 27027
Sun Jul 28 09:07:44.638 [initandlisten] connection accepted from 127.0.0.1:42668 #1 (1 connection now open)
Sun Jul 28 09:07:44.657 [conn1] ERROR:   mmap() failed for /home/ravi/standalone/bhat.7 len:536608768 errno:12 Cannot allocate memory
Sun Jul 28 09:07:44.657 [conn1] ERROR: mmap failed with out of memory. You are using a 32-bit build and probably need to upgrade to 64
Sun Jul 28 09:07:44.662 [conn1] ERROR:   mmap() failed for /home/ravi/standalone/coke.4 len:268435456 errno:12 Cannot allocate memory
Sun Jul 28 09:07:44.662 [conn1] ERROR: mmap failed with out of memory. You are using a 32-bit build and probably need to upgrade to 64
Sun Jul 28 09:07:44.700 [initandlisten] connection accepted from 127.0.0.1:42669 #2 (2 connections now open)
Sun Jul 28 09:07:44.708 [conn2]  authenticate db: admin { authenticate: 1, user: "pakhandi", nonce: "b83c897df2baa4b2", key: "73f4bf89cda7ffde3953e89e56a63a9d" }
Sun Jul 28 09:07:44.829 [initandlisten] connection accepted from 127.0.0.1:42670 #3 (3 connections now open)
Sun Jul 28 09:07:44.832 [conn3]  authenticate db: admin { authenticate: 1, user: "pakhandi", nonce: "af275ed53d2d86d9", key: "3190c773b60b2d48d54d55f58021034b" }
Sun Jul 28 09:07:49.367 [initandlisten] connection accepted from 127.0.0.1:42671 #4 (4 connections now open)
Sun Jul 28 09:08:18.440 [conn3] error: getFile() called in a read lock, yet file to return is not yet open
Sun Jul 28 09:08:18.440 [conn3]        getFile(7) _files.size:7 /home/ravi/standalone/bhat.7
Sun Jul 28 09:08:18.440 [conn3]        context ns: bhat.genome
Sun Jul 28 09:08:18.440 [conn3]  bhat.genome Assertion failure false src/mongo/db/database.cpp 300
0x89f79fd 0x89cd525 0x89b298e 0x84cb30c 0x86cc113 0x8763bfa 0x8767277 0x8767a46 0x8683d44 0x8686b4f 0x85f8999 0x85f9e15 0x82d8175 0x89e26f0 0x8ac96e 0x37d3fe 
 mongod(_ZN5mongo15printStackTraceERSo+0x2d) [0x89f79fd]
 mongod(_ZN5mongo10logContextEPKc+0xa5) [0x89cd525]
 mongod(_ZN5mongo12verifyFailedEPKcS1_j+0x12e) [0x89b298e]
 mongod(_ZN5mongo8Database7getFileEiib+0x34c) [0x84cb30c]
 mongod(_ZN5mongo11DataFileMgr7findAllERKNS_10StringDataERKNS_7DiskLocE+0x393) [0x86cc113]
 mongod(_ZNK5mongo15CursorGenerator14shortcutCursorEv+0x22a) [0x8763bfa]
 mongod(_ZN5mongo15CursorGenerator8generateEv+0x37) [0x8767277]
 mongod(_ZN5mongo25NamespaceDetailsTransient9getCursorERKNS_10StringDataERKNS_7BSONObjES6_RKNS_24QueryPlanSelectionPolicyERKN5boost10shared_ptrIKNS_11ParsedQueryEEEbPNS_16QueryPlanSummaryE+0x66) [0x8767a46]
 mongod(_ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_12ChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE+0xf4) [0x8683d44]
 mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x183f) [0x8686b4f]
 mongod() [0x85f8999]
 mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x3f5) [0x85f9e15]
 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x85) [0x82d8175]
 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x490) [0x89e26f0]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x8ac96e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0x37d3fe]
Sun Jul 28 09:08:18.448 [conn3] assertion 0 assertion src/mongo/db/database.cpp:300 ns:bhat.genome query:{}
Sun Jul 28 09:08:18.448 [conn3]  ntoskip:0 ntoreturn:-1

The problem is here:

Sun Jul 28 09:07:44.657 [conn1] ERROR:   mmap() failed for /home/ravi/standalone/bhat.7 len:536608768 errno:12 Cannot allocate memory
Sun Jul 28 09:07:44.657 [conn1] ERROR: mmap failed with out of memory. You are using a 32-bit build and probably need to upgrade to 64
Sun Jul 28 09:07:44.662 [conn1] ERROR:   mmap() failed for /home/ravi/standalone/coke.4 len:268435456 errno:12 Cannot allocate memory
Sun Jul 28 09:07:44.662 [conn1] ERROR: mmap failed with out of memory. You are using a 32-bit build and probably need to upgrade to 64

32-bit versions are only able to access a limited amount of data because of the memory-mapping method that MongoDB uses to access database files. In the log, mongod is saying that it cannot memory map ("mmap()") your files due to an inability to allocate memory to hold them. You will need to either remove data or switch to a 64-bit version of MongoDB (which will require a 64-bit OS).

Comment by Ravi Shakya [ 28/Jul/13 ]

mongod server log file

Comment by Ravi Shakya [ 28/Jul/13 ]

Hi Tad,
Thanks for the prompt response. Please find attached the mongod server log file.

Comment by Tad Marshall [ 27/Jul/13 ]

Hi Ravi,

Your screen shot only shows the client side of the problem. The server log will have more information on what is going wrong.

The assertion at line 300 of database.cpp follows some log lines saying what the problem is and will be followed in the log by a stack trace showing how this code was called. We would want to see the entire log for clues.

This is the code that is generating the assertion failure:

        if ( !preallocateOnly ) {
            while ( n >= (int) _files.size() ) {
                verify(this);
                if( !Lock::isWriteLocked(this->name) ) {
                    log() << "error: getFile() called in a read lock, yet file to return is not yet open" << endl;
                    log() << "       getFile(" << n << ") _files.size:" <<_files.size() << ' ' << fileName(n).string() << endl;
                    log() << "       context ns: " << cc().ns() << endl;
                    verify(false);
                }
                _files.push_back(0);
            }
            p = _files[n];
        }

The "verify(false);" line above is line 300 in database.cpp in version 2.4.2.

Can you attach a full log (from the start of mongod) to this ticket? A full log of the mongo shell session would also be helpful so that we can tie events on the client with events on the server.

Tad

Generated at Thu Feb 08 03:22:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.