[SERVER-2631] Assertion failure a <= 128*1024*1024 util/alignedbuilder.cpp 51 with durability Created: 27/Feb/11  Updated: 12/Jul/16  Resolved: 02/Mar/11

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 1.7.4, 1.7.5, 1.7.6, 1.8.0-rc0
Fix Version/s: 1.8.0-rc1

Type: Bug Priority: Major - P3
Reporter: Swen Thümmler Assignee: Dwight Merriman
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

linux 64bit


Attachments: File crash.bson.bz2    
Operating System: Linux
Participants:

 Description   

When trying to repair a replica set member, i ran into this assertion. You may reproduce this issue with the attached collection.
Load it with mongorestore -d test -c crash --drop x and then use reindex. If you started the server with --dur, the reindex will lead to the assertion, if you started it without --dur, reindex will work.



 Comments   
Comment by auto [ 01/Mar/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing commitIfNeeded in ~BtreeBuilder - 180?
https://github.com/mongodb/mongo/commit/22596b3c06db48660dd3eec96ca833ab584136ad

Comment by auto [ 01/Mar/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing destructor guard - 180?
https://github.com/mongodb/mongo/commit/d328f5b1a5bcc021ccf8c2c96acab0e2eca0aeb6

Comment by auto [ 01/Mar/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing commitIfNeeded call - 180?
https://github.com/mongodb/mongo/commit/09ed6d9098b991ee36ea3cb3ee5232f3e50b84ab

Comment by auto [ 01/Mar/11 ]

Author:

{u'login': u'dwight', u'name': u'dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 180? 181?
https://github.com/mongodb/mongo/commit/b2963ff3b293dc5eaf9acaf6de5b4fc835beb6b2

Comment by auto [ 28/Feb/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing commitIfNeeded in ~BtreeBuilder - 180?
https://github.com/mongodb/mongo/commit/95e224e65f5ea80e3aa8f65f66cd4d4e1bafb3ef

Comment by auto [ 28/Feb/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing destructor guard - 180?
https://github.com/mongodb/mongo/commit/81d0154d41c1c8416cfbe0f423614edca4834e36

Comment by auto [ 28/Feb/11 ]

Author:

{u'login': u'dwight', u'name': u'Dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 missing commitIfNeeded call - 180?
https://github.com/mongodb/mongo/commit/f46d86c254f2ade330de5af4d80d174b6c51f2bc

Comment by Swen Thümmler [ 27/Feb/11 ]

1.7.4 does crash, too (with a different assertion).

Comment by Swen Thümmler [ 27/Feb/11 ]

Nope, it is not just a warning, the server does crash. Here is the server log:
swen@gasherbrum </opt>: /opt/mongodb/bin/mongod --dbpath /var/tmp/mongo --port 10000 --dur
Sun Feb 27 17:46:43 [initandlisten] MongoDB starting : pid=21893 port=10000 dbpath=/var/tmp/mongo 64-bit
Sun Feb 27 17:46:43 [initandlisten] db version v1.8.0-rc0, pdfile version 4.5
Sun Feb 27 17:46:43 [initandlisten] git version: 65a7e81df0747b6bc9380b78e0182192bacdb4d0
Sun Feb 27 17:46:43 [initandlisten] build sys info: Linux ip-10-110-9-236 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Sun Feb 27 17:46:43 [initandlisten] journal dir=/var/tmp/mongo/journal
Sun Feb 27 17:46:43 [initandlisten] recover : no journal files present, no recovery needed
Sun Feb 27 17:46:47 [initandlisten] preallocateIsFaster=true 42.86
Sun Feb 27 17:46:50 [initandlisten] preallocateIsFaster=true 44.74
Sun Feb 27 17:46:54 [initandlisten] preallocateIsFaster=true 43.86
Sun Feb 27 17:46:54 [initandlisten] waiting for connections on port 10000
Sun Feb 27 17:46:54 [websvr] web admin interface listening on port 11000
Sun Feb 27 17:46:57 [initandlisten] connection accepted from 127.0.0.1:46474 #1
Sun Feb 27 17:47:11 [conn1] CMD: reIndex test.crash
Sun Feb 27 17:47:11 [conn1] building new index on

{ _id: 1 }

for test.crash
2740300/6414289 42%
4237700/6414289 66%
6000000/6414289 93%
Sun Feb 27 17:47:43 [conn1] external sort used : 7 files in 32 secs
3887600/6414289 60%
Sun Feb 27 17:48:03 [conn1] done building bottom layer, going to commit
Sun Feb 27 17:48:03 terminate() called, printing stack:
0x8a241d 0x7f8ba155d906 0x7f8ba155c8d9 0x7f8ba155d1f5 0x7f8ba101a49b 0x7f8ba101a90b 0x67ebae 0x776dbc 0x777318 0x778d70 0x77a1ac 0x7e660e 0x7dbc52 0x7dcb95 0x644c55 0x649191 0x757995 0x759ec0 0x8a65ee 0x8b6a30
/opt/mongodb/bin/mongod(_ZN5mongo11myterminateEv+0x4d) [0x8a241d]
/usr/lib/libstdc++.so.6(+0xb9906) [0x7f8ba155d906]
/usr/lib/libstdc++.so.6(+0xb88d9) [0x7f8ba155c8d9]
/usr/lib/libstdc++.so.6(__gxx_personality_v0+0x235) [0x7f8ba155d1f5]
/lib/libgcc_s.so.1(+0xf49b) [0x7f8ba101a49b]
/lib/libgcc_s.so.1(_Unwind_Resume+0x8b) [0x7f8ba101a90b]
/opt/mongodb/bin/mongod(_ZN5mongo12BtreeBuilderD1Ev+0x63e) [0x67ebae]
/opt/mongodb/bin/mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x17bc) [0x776dbc]
/opt/mongodb/bin/mongod() [0x777318]
/opt/mongodb/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0xf60) [0x778d70]
/opt/mongodb/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb+0x4c) [0x77a1ac]
/opt/mongodb/bin/mongod(_ZN5mongo10CmdReIndex3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x6de) [0x7e660e]
/opt/mongodb/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x9e2) [0x7dbc52]
/opt/mongodb/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x895) [0x7dcb95]
/opt/mongodb/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x35) [0x644c55]
/opt/mongodb/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x32e1) [0x649191]
/opt/mongodb/bin/mongod() [0x757995]
/opt/mongodb/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0x5b0) [0x759ec0]
/opt/mongodb/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x21e) [0x8a65ee]
/opt/mongodb/bin/mongod(thread_proxy+0x80) [0x8b6a30]
Sun Feb 27 17:48:03 Got signal: 6 (Aborted).

Sun Feb 27 17:48:03 Backtrace:
0x8a3189 0x7f8ba0cbbc20 0x7f8ba0cbbba5 0x7f8ba0cbf6b0 0x8a24eb 0x7f8ba155d906 0x7f8ba155c8d9 0x7f8ba155d1f5 0x7f8ba101a49b 0x7f8ba101a90b 0x67ebae 0x776dbc 0x777318 0x778d70 0x77a1ac 0x7e660e 0x7dbc52 0x7dcb95 0x644c55 0x649191
/opt/mongodb/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x8a3189]
/lib/libc.so.6(+0x33c20) [0x7f8ba0cbbc20]
/lib/libc.so.6(gsignal+0x35) [0x7f8ba0cbbba5]
/lib/libc.so.6(abort+0x180) [0x7f8ba0cbf6b0]
/opt/mongodb/bin/mongod(_ZN5mongo11myterminateEv+0x11b) [0x8a24eb]
/usr/lib/libstdc++.so.6(+0xb9906) [0x7f8ba155d906]
/usr/lib/libstdc++.so.6(+0xb88d9) [0x7f8ba155c8d9]
/usr/lib/libstdc++.so.6(__gxx_personality_v0+0x235) [0x7f8ba155d1f5]
/lib/libgcc_s.so.1(+0xf49b) [0x7f8ba101a49b]
/lib/libgcc_s.so.1(_Unwind_Resume+0x8b) [0x7f8ba101a90b]
/opt/mongodb/bin/mongod(_ZN5mongo12BtreeBuilderD1Ev+0x63e) [0x67ebae]
/opt/mongodb/bin/mongod(_ZN5mongo14fastBuildIndexEPKcPNS_16NamespaceDetailsERNS_12IndexDetailsEi+0x17bc) [0x776dbc]
/opt/mongodb/bin/mongod() [0x777318]
/opt/mongodb/bin/mongod(_ZN5mongo11DataFileMgr6insertEPKcPKvibRKNS_11BSONElementEb+0xf60) [0x778d70]
/opt/mongodb/bin/mongod(_ZN5mongo11DataFileMgr16insertWithObjModEPKcRNS_7BSONObjEb+0x4c) [0x77a1ac]
/opt/mongodb/bin/mongod(_ZN5mongo10CmdReIndex3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x6de) [0x7e660e]
/opt/mongodb/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x9e2) [0x7dbc52]
/opt/mongodb/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x895) [0x7dcb95]
/opt/mongodb/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x35) [0x644c55]
/opt/mongodb/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0x32e1) [0x649191]

Sun Feb 27 17:48:03 dbexit:
Sun Feb 27 17:48:03 [conn1] shutdown: going to close listening sockets...
Sun Feb 27 17:48:03 [conn1] closing listening socket: 5
Sun Feb 27 17:48:03 [conn1] closing listening socket: 6
Sun Feb 27 17:48:03 [conn1] closing listening socket: 7
Sun Feb 27 17:48:03 [conn1] closing listening socket: 8
Sun Feb 27 17:48:03 [conn1] removing socket file: /tmp/mongodb-10000.sock
Sun Feb 27 17:48:03 [conn1] removing socket file: /tmp/mongodb-11000.sock
Sun Feb 27 17:48:03 [conn1] shutdown: going to flush diaglog...
Sun Feb 27 17:48:03 [conn1] shutdown: going to close sockets...
Sun Feb 27 17:48:03 [conn1] shutdown: waiting for fs preallocator...
Sun Feb 27 17:48:03 [conn1] shutdown: lock for final commit...
Sun Feb 27 17:48:03 [conn1] shutdown: final commit...
Sun Feb 27 17:48:03 [conn1] test Assertion failure a <= 128*1024*1024 util/alignedbuilder.cpp 51
0x55125e 0x5628af 0x629154 0x636d1a 0x637574 0x637729 0x6378fb 0x62e1bf 0x62eb48 0x753b86 0x754454 0x8a32d5 0x7f8ba0cbbc20 0x7f8ba0cbbba5 0x7f8ba0cbf6b0 0x8a24eb 0x7f8ba155d906 0x7f8ba155c8d9 0x7f8ba155d1f5 0x7f8ba101a49b
/opt/mongodb/bin/mongod(_ZN5mongo12sayDbContextEPKc+0xae) [0x55125e]
/opt/mongodb/bin/mongod(_ZN5mongo9wassertedEPKcS1_j+0x10f) [0x5628af]
/opt/mongodb/bin/mongod(_ZN5mongo14AlignedBuilder14growReallocateEj+0x44) [0x629154]
/opt/mongodb/bin/mongod(_ZN5mongo3dur21prepBasicWrite_inlockERNS_14AlignedBuilderEPKNS0_11WriteIntentERNS_12RelativePathE+0x2aa) [0x636d1a]
/opt/mongodb/bin/mongod(_ZN5mongo3dur15prepBasicWritesERNS_14AlignedBuilderE+0x74) [0x637574]
/opt/mongodb/bin/mongod(_ZN5mongo3dur14_PREPLOGBUFFEREv+0x79) [0x637729]
/opt/mongodb/bin/mongod(_ZN5mongo3dur13PREPLOGBUFFEREv+0x9b) [0x6378fb]
/opt/mongodb/bin/mongod() [0x62e1bf]
/opt/mongodb/bin/mongod(_ZN5mongo3dur11DurableImpl9commitNowEv+0x18) [0x62eb48]
/opt/mongodb/bin/mongod(_ZN5mongo14shutdownServerEv+0x466) [0x753b86]
/opt/mongodb/bin/mongod(_ZN5mongo6dbexitENS_8ExitCodeEPKcb+0x1f4) [0x754454]
/opt/mongodb/bin/mongod(_ZN5mongo10abruptQuitEi+0x4e5) [0x8a32d5]
/lib/libc.so.6(+0x33c20) [0x7f8ba0cbbc20]
/lib/libc.so.6(gsignal+0x35) [0x7f8ba0cbbba5]
/lib/libc.so.6(abort+0x180) [0x7f8ba0cbf6b0]
/opt/mongodb/bin/mongod(_ZN5mongo11myterminateEv+0x11b) [0x8a24eb]
/usr/lib/libstdc++.so.6(+0xb9906) [0x7f8ba155d906]
/usr/lib/libstdc++.so.6(+0xb88d9) [0x7f8ba155c8d9]
/usr/lib/libstdc++.so.6(__gxx_personality_v0+0x235) [0x7f8ba155d1f5]
/lib/libgcc_s.so.1(+0xf49b) [0x7f8ba101a49b]
Sun Feb 27 17:48:13 [conn1] shutdown: closing all files...
Sun Feb 27 17:48:13 closeAllFiles() finished
Sun Feb 27 17:48:13 [conn1] shutdown: journalCleanup...
Sun Feb 27 17:48:13 [conn1] removeJournalFiles
Sun Feb 27 17:48:13 [conn1] shutdown: removing fs lock...
Sun Feb 27 17:48:13 dbexit: really exiting now
Sun Feb 27 17:48:13 ERROR: Client::~Client _context should be null but is not; client:conn
[1] 21893 exit 14 /opt/mongodb/bin/mongod --dbpath /var/tmp/mongo --port 10000 --dur

Comment by auto [ 27/Feb/11 ]

Author:

{u'login': u'dwight', u'name': u'dwight', u'email': u'dwight@10gen.com'}

Message: SERVER-2631 180? 181?
https://github.com/mongodb/mongo/commit/733134f7da8c67f35096833539a101493968e84f

Comment by Dwight Merriman [ 27/Feb/11 ]

i'm pretty sure that is just a "warning assert" and it keeps going.

but the threshold is too low, it shouldn't be firing at all, will clean it up.

Generated at Thu Feb 08 03:00:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.