[SERVER-14693] Indexing bigass fields makes mongod sad panda Created: 25/Jul/14  Updated: 10/Dec/14  Resolved: 19/Aug/14

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 2.6.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Hannes Magnusson Assignee: Ramon Fernandez Marina
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

fetch http://mongodb-enron-email.s3-website-us-east-1.amazonaws.com/

$ mongod --setParameter failIndexKeyTooLong=0
$ mongorestore enron_mail
$ mongo
MongoDB shell version: 2.6.3
connecting to: test
> use enron_mail
switched to db enron_mail
> db.messages.createIndex({"body": 1}, { type: "sparse"})

Participants:

 Description   

2014-07-25T21:28:12.010+0000 [conn1] enron_mail.messages Btree::insert: key too large to index, failing enron_mail.messages.$body_1 1184 { : "Dudes:
I'm am officially responsible for the case and of course appreciate all
Dylan's (and everyone else's) efforts.  Here's my game plan.  I'm goin..." }
2014-07-25T21:28:12.010+0000 [conn1] enron_mail.messages Btree::insert: key too large to index, failing enron_mail.messages.$body_1 1808 { : "Due to Ramona being out Friday, I would like to keep this CAS overview scheduled for today, Thursday afternoon, from 1:30 to 3pm.  Ramona will be avai..." }
2014-07-25T21:28:12.470+0000 [conn1] Assertion: 13524:out of memory AlignedBuilder
2014-07-25T21:28:12.497+0000 [conn1] enron_mail.messages 0x11da801 0x117c9f9 0x11614b6 0x115eabc 0x115eae9 0xa6920f 0xa694c4 0xa5bee7 0xa5c58a 0xa5997f 0xa59a36 0xf1392a 0xf159f1 0xb69071 0xb63307 0x8e99f8 0x8e0f20 0x931e8b 0xa2295a 0xa2431b
 mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11da801]
 mongod(_ZN5mongo10logContextEPKc+0x159) [0x117c9f9]
 mongod(_ZN5mongo11msgassertedEiPKc+0xe6) [0x11614b6]
 mongod() [0x115eabc]
 mongod(_ZN5mongo14AlignedBuilder8_reallocEjj+0x29) [0x115eae9]
 mongod() [0xa6920f]
 mongod(_ZN5mongo3dur13PREPLOGBUFFERERNS0_11JSectHeaderERNS_14AlignedBuilderE+0x214) [0xa694c4]
 mongod() [0xa5bee7]
 mongod(_ZN5mongo3dur11DurableImpl9commitNowEv+0x1a) [0xa5c58a]
 mongod(_ZN5mongo3dur11DurableImpl16_aCommitIsNeededEv+0x25f) [0xa5997f]
 mongod(_ZN5mongo3dur11DurableImpl14commitIfNeededEb+0x56) [0xa59a36]
 mongod(_ZN5mongo12BtreeBuilderINS_12BtreeData_V1EE24mayCommitProgressDurablyEv+0x1a) [0xf1392a]
 mongod(_ZN5mongo12BtreeBuilderINS_12BtreeData_V1EE6addKeyERNS_7BSONObjENS_7DiskLocE+0x111) [0xf159f1]
 mongod(_ZN5mongo9BtreeBulk6commitINS_12BtreeData_V1EEEvPSt3setINS_7DiskLocESt4lessIS4_ESaIS4_EEPNS_5CurOpEb+0x1a1) [0xb69071]
 mongod(_ZN5mongo22BtreeBasedAccessMethod10commitBulkEPNS_17IndexAccessMethodEbPSt3setINS_7DiskLocESt4lessIS4_ESaIS4_EE+0x207) [0xb63307]
 mongod(_ZN5mongo12buildAnIndexEPNS_10CollectionEPNS_17IndexCatalogEntryEb+0x9b8) [0x8e99f8]
 mongod(_ZN5mongo12IndexCatalog11createIndexENS_7BSONObjEbNS0_16ShutdownBehaviorE+0x6e0) [0x8e0f20]
 mongod(_ZN5mongo14CmdCreateIndex3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x24fb) [0x931e8b]
 mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2295a]
 mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xfeb) [0xa2431b]
2014-07-25T21:28:12.503+0000 [conn1] dbexception in groupCommit causing immediate shutdown: 13524 out of memory AlignedBuilder
2014-07-25T21:28:12.503+0000 [conn1] SEVERE: gc1
2014-07-25T21:28:12.517+0000 [conn1] SEVERE: Got signal: 6 (Aborted).
Backtrace:0x11da801 0x11d9bde 0x7ff14661eff0 0x7ff14661ef79 0x7ff146622388 0xb9180b 0xa5c32d 0xa5c58a 0xa5997f 0xa59a36 0xf1392a 0xf159f1 0xb69071 0xb63307 0x8e99f8 0x8e0f20 0x931e8b 0xa2295a 0xa2431b 0xa26269
 mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11da801]
 mongod() [0x11d9bde]
 /lib/x86_64-linux-gnu/libc.so.6(+0x36ff0) [0x7ff14661eff0]
 /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7ff14661ef79]
 /lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7ff146622388]
 mongod(_ZN5mongo10mongoAbortEPKc+0x6b) [0xb9180b]
 mongod() [0xa5c32d]
 mongod(_ZN5mongo3dur11DurableImpl9commitNowEv+0x1a) [0xa5c58a]
 mongod(_ZN5mongo3dur11DurableImpl16_aCommitIsNeededEv+0x25f) [0xa5997f]
 mongod(_ZN5mongo3dur11DurableImpl14commitIfNeededEb+0x56) [0xa59a36]
 mongod(_ZN5mongo12BtreeBuilderINS_12BtreeData_V1EE24mayCommitProgressDurablyEv+0x1a) [0xf1392a]
 mongod(_ZN5mongo12BtreeBuilderINS_12BtreeData_V1EE6addKeyERNS_7BSONObjENS_7DiskLocE+0x111) [0xf159f1]
 mongod(_ZN5mongo9BtreeBulk6commitINS_12BtreeData_V1EEEvPSt3setINS_7DiskLocESt4lessIS4_ESaIS4_EEPNS_5CurOpEb+0x1a1) [0xb69071]
 mongod(_ZN5mongo22BtreeBasedAccessMethod10commitBulkEPNS_17IndexAccessMethodEbPSt3setINS_7DiskLocESt4lessIS4_ESaIS4_EE+0x207) [0xb63307]
 mongod(_ZN5mongo12buildAnIndexEPNS_10CollectionEPNS_17IndexCatalogEntryEb+0x9b8) [0x8e99f8]
 mongod(_ZN5mongo12IndexCatalog11createIndexENS_7BSONObjEbNS0_16ShutdownBehaviorE+0x6e0) [0x8e0f20]
 mongod(_ZN5mongo14CmdCreateIndex3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x24fb) [0x931e8b]
 mongod(_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x3a) [0xa2295a]
 mongod(_ZN5mongo7Command11execCommandEPS0_RNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xfeb) [0xa2431b]
 mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x6c9) [0xa26269]



 Comments   
Comment by Ramon Fernandez Marina [ 19/Aug/14 ]

bjori, I'm going to close this issue as "can't reproduce". As for the error message you're seeing, this is the way things are today for better or worse: when mongod finds a critical problem it shuts itself down to protect users' data. When this happens the user will see a Fatal Assertion in the logs followed by a stack trace, and in many cases a subsequent abort.

Regards,
Ramón.

Comment by Ramon Fernandez Marina [ 29/Jul/14 ]

bjori, I'm not able to reproduce the problem, even with failIndexKeyTooLong=0: warning about long keys are sent to the logs, but no to the shell as expected.

In the log you sent I see:

2014-07-25T21:28:12.470+0000 [conn1] Assertion: 13524:out of memory AlignedBuilder

so I suspect this is not really an issue with indexing. I agree that it's not ideal to have mongod shut down, but if the shutdown was caused by incorrect permissions I can't seem to reproduce that either – when I do chmod 000 journal the log says (this is on MacOS):

2014-07-29T11:14:43.646-0400 [journal] In File::open(), ::open for './db/journal/lsn' failed with errno:13 Permission denied

Generated at Thu Feb 08 03:35:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.