[SERVER-19277] exception: BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO and can not be repaired Created: 03/Jul/15  Updated: 03/Aug/15  Resolved: 03/Aug/15

Status: Closed
Project: Core Server
Component/s: Index Maintenance, Storage
Affects Version/s: 3.0.4
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Bin He Assignee: Sam Kleinman (Inactive)
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:

 Description   

I use mongodb to store about 400GB data. And when I tried to ensureIndex, it exit and I found the backtrace in the log:

015-07-03T02:23:09.638+0800 I CONTROL  [conn3]
 0xf7a0f9 0xf19121 0xefe28f 0xefe33c 0x851b4f 0x931422 0xa19793 0xbec614 0x9494ab 0x971dde 0x9f40e4 0x9f506d 0x9f5d7b 0xbbabca 0xad0f00 0x82b88d 0xf2d22b 0x7fe77aa6fb50 0x7fe7795b6e6d
----- BEGIN BACKTRACE -----
{"backtrace":[{"b":"400000","o":"B7A0F9"},{"b":"400000","o":"B19121"},{"b":"400000","o":"AFE28F"},{"b":"400000","o":"AFE33C"},{"b":"400000","o":"451B4F"},{"b":"400000","o":"531422"},{"b":"400000","o":"619793"},{"b":"400000","o":"7EC614"},{"b":"400000","o":"5494AB"},{"b":"400000","o":"571DDE"},{"b":"400000","o":"5F40E4"},{"b":"400000","o":"5F506D"},{"b":"400000","o":"5F5D7B"},{"b":"400000","o":"7BABCA"},{"b":"400000","o":"6D0F00"},{"b":"400000","o":"42B88D"},{"b":"400000","o":"B2D22B"},{"b":"7FE77AA69000","o":"6B50"},{"b":"7FE7794DB000","o":"DBE6D"}],"processInfo":{ "mongodbVersion" : "3.0.4", "gitVersion" : "0481c958daeb2969800511e7475dc66986fa9ed5", "uname" : { "sysname" : "Linux", "release" : "3.2.0-4-amd64", "version" : "#1 SMP Debian 3.2.60-1+deb7u3", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "400000", "buildId" : "EB7A0F7D9F202DB3ADD7637D20509D9DE82AC5A1" }, { "b" : "7FFFAECB0000", "elfType" : 3, "buildId" : "804F357CF825E4716BA7DA60414184939D1D3432" }, { "b" : "7FE77AA69000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "E2755F1148137BA124B24BF1072EF146FE51F361" }, { "b" : "7FE77A809000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "90B85CC62C41AE65576EA31156205F149B04FD31" }, { "b" : "7FE77A411000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "9CC15590792B6F822DBB2ACEF66CB9A89FAA0775" }, { "b" : "7FE77A209000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "514F17B70AF2A2DBC501771B1271138EE35FF3AB" }, { "b" : "7FE77A005000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D0015E317E2EEDCC8F37457A504A1746DE763C49" }, { "b" : "7FE779CFE000", "path" : "/usr/lib/x86_64-linux-gnu/libstdc++.so.6", "elfType" : 3, "buildId" : "8711429397A5AF8B6269B867D830EDF6E0225B8D" }, { "b" : "7FE779A7C000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "9233B6861A56025E3379F1FF173C230E66E39B5E" }, { "b" : "7FE779866000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "F980B1188708F8D8B5C35D185444AF4CB939AA1E" }, { "b" : "7FE7794DB000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "A23A05B5AF000742B37A68C9D8D5A857D01BCB9F" }, { "b" : "7FE77AC85000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "A16192716DE58BA171CDA81D39F045BF24849176" }, { "b" : "7FE7792C4000", "path" : "/lib/x86_64-linux-gnu/libz.so.1", "elfType" : 3, "buildId" : "1EFEB71FD4999C2307570D673A724EA4E1D85267" } ] }}
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf7a0f9]
 mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf19121]
 mongod(_ZN5mongo11msgassertedEiPKc+0xAF) [0xefe28f]
 mongod(+0xAFE33C) [0xefe33c]
 mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x3DF) [0x851b4f]
 mongod(_ZN5mongo10RecordData13releaseToBsonEv+0x52) [0x931422]
 mongod(_ZN5mongo14CollectionScan4workEPm+0x233) [0xa19793]
 mongod(_ZN5mongo12PlanExecutor18getNextSnapshottedEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE+0xA4) [0xbec614]
 mongod(_ZN5mongo15MultiIndexBlock30insertAllDocumentsInCollectionEPSt3setINS_8RecordIdESt4lessIS2_ESaIS2_EE+0x17B) [0x9494ab]
 mongod(_ZN5mongo14CmdCreateIndex3runEPNS_16OperationContextERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x104E) [0x971dde]
 mongod(_ZN5mongo12_execCommandEPNS_16OperationContextEPNS_7CommandERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x34) [0x9f40e4]
 mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_iPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xC1D) [0x9f506d]
 mongod(_ZN5mongo12_runCommandsEPNS_16OperationContextEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x28B) [0x9f5d7b]
 mongod(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_+0x77A) [0xbbabca]
 mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xB10) [0xad0f00]
 mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xDD) [0x82b88d]
 mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x34B) [0xf2d22b]
 libpthread.so.0(+0x6B50) [0x7fe77aa6fb50]
 libc.so.6(clone+0x6D) [0x7fe7795b6e6d]
-----  END BACKTRACE  -----

the error_msg is "exception: BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO"

mongodump --repair and db.repairDatabase() does not work.

How to restore the data as many as possible?



 Comments   
Comment by Sam Kleinman (Inactive) [ 07/Jul/15 ]

Thanks for this report, and sorry that you've run into this issue. Could you please answer the following questions about your deployment:

  1. Which storage engine are you using?
  2. Can you provide more information about the underlying storage system? SSDs or HDDs? Local or network storage? What kind of filesystem is configured?
  3. What output did mongodump --repair and db.repairDatabase() return when they did not work?
  4. How did you insert data into this database? Which driver did you use?
  5. Was this the first index you attempted to build? Are there other existing indexes. Could you provide an example document, and the output of db.getIndexes()?

Regards,
sam

Generated at Thu Feb 08 03:50:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.