[SERVER-8851] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 Created: 05/Mar/13  Updated: 10/Dec/14  Resolved: 28/Oct/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.2.3
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Jonathan Travin Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: bson, core, corrupt
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS 6.3


Operating System: ALL
Participants:

 Description   

We have a replicaset running on 2 servers (+1 arbiter), but in time to time we start getting the assertion 10334 error without any modifications.
Currently our way to solve the issue is to remove the primary server (switching automatically to the secondary one), and start from scratch the syncing to add again the replicaset.
This works, but also is not at all scalable and not something that is useful.
When this error starts, it's not for a particular row, but any new updated row in any of the collections (we have only one database).

Primary mongo.log

Tue Mar  5 09:56:29 [conn311369] update fbridge.activity_feed query: { _id: ObjectId('511e4130a298d8d31600071e') } update: { user_id: 186564, created_at: new Date(1360937264084), updated_at: new Date(1362495389507), network: "youtube", source: "total_views", external_id: "9223372036854775807", data: { username: "jayz", url: "https://www.youtube.com/channel/UCN-sc1xJr-QQNj_uNIM9wTA", profile_pic: "https://i3.ytimg.com/i/N-sc1xJr-QQNj_uNIM9wTA/1.jpg?v=506e1b60", channel_id: "UCN-sc1xJr-QQNj_uNIM9wTA", user_id: "N-sc1xJr-QQNj_uNIM9wTA", title: "profile stats", importance: 21, total: 346413303 }, activities: [ { total: 345112999, datetime: new Date(1362053676294) }, { total: 345113211, datetime: new Date(1362060046295) }, { total: 345158210, datetime: new Date(1362066608364) }, { total: 345174676, datetime: new Date(1362069756052) }, { total: 345177823, datetime: new Date(1362072928729) }, { total: 345189802, datetime: new Date(1362079298240) }, { total: 345191299, datetime: new Date(1362085671950) }, { total: 345214768, datetime: new Date(1362095423013) }, { total: 345214953, datetime: new Date(1362098570659) }, { total: 345245828, datetime: new Date(1362101745768) }, { total: 345250603, datetime: new Date(1362104925365) }, { total: 345253486, datetime: new Date(1362108080900) }, { total: 345290143, datetime: new Date(1362117651783) }, { total: 345302522, datetime: new Date(1362121018881) }, { total: 345334617, datetime: new Date(1362124196967) }, { total: 345342162, datetime: new Date(1362127742719) }, { total: 345347896, datetime: new Date(1362130922444) }, { total: 345377353, datetime: new Date(1362134105852) }, { total: 345393550, datetime: new Date(1362137284890) }, { total: 345408136, datetime: new Date(1362146824899) }, { total: 345416329, datetime: new Date(1362150003392) }, { total: 345431641, datetime: new Date(1362156546651) }, { total: 345439207, datetime: new Date(1362159727780) }, { total: 345439307, datetime: new Date(1362162907416) }, { total: 345453340, datetime: new Date(1362166087432) }, { total: 345459572, datetime: new Date(1362169266274) }, { total: 345459604, datetime: new Date(1362175627141) }, { total: 345487907, datetime: new Date(1362178805641) }, { total: 345501425, datetime: new Date(1362182169302) }, { total: 345530937, datetime: new Date(1362188525718) }, { total: 345543997, datetime: new Date(1362191704769) }, { total: 345578692, datetime: new Date(1362198071059) }, { total: 345587409, datetime: new Date(1362204426821) }, { total: 345620199, datetime: new Date(1362210969752) }, { total: 345628177, datetime: new Date(1362214149516) }, { total: 345648694, datetime: new Date(1362220508954) }, { total: 345654209, datetime: new Date(1362223688273) }, { total: 345683776, datetime: new Date(1362233231826) }, { total: 345693756, datetime: new Date(1362236412325) }, { total: 345693844, datetime: new Date(1362239768187) }, { total: 345712253, datetime: new Date(1362242950823) }, { total: 345720069, datetime: new Date(1362246131435) }, { total: 345720168, datetime: new Date(1362249314338) }, { total: 345737934, datetime: new Date(1362252491362) }, { total: 345744975, datetime: new Date(1362255669723) }, { total: 345773190, datetime: new Date(1362265208480) }, { total: 345786343, datetime: new Date(1362268565761) }, { total: 345817953, datetime: new Date(1362274929301) }, { total: 345818396, datetime: new Date(1362278110393) }, { total: 345828157, datetime: new Date(1362281287184) }, { total: 345838128, datetime: new Date(1362284471147) }, { total: 345839445, datetime: new Date(1362290823824) }, { total: 345843025, datetime: new Date(1362294018386) }, { total: 345900246, datetime: new Date(1362297369007) }, { total: 345910111, datetime: new Date(1362300978751) }, { total: 345937570, datetime: new Date(1362308172725) }, { total: 345944584, datetime: new Date(1362311771273) }, { total: 345944652, datetime: new Date(1362315369671) }, { total: 345960847, datetime: new Date(1362318974098) }, { total: 345964878, datetime: new Date(1362322576553) }, { total: 345964970, datetime: new Date(1362326173992) }, { total: 345980870, datetime: new Date(1362329773503) }, { total: 345983468, datetime: new Date(1362333373759) }, { total: 345983568, datetime: new Date(1362336979005) }, { total: 346000246, datetime: new Date(1362340576749) }, { total: 346001927, datetime: new Date(1362344186565) }, { total: 346028843, datetime: new Date(1362351365920) }, { total: 346032100, datetime: new Date(1362354983101) }, { total: 346059841, datetime: new Date(1362362188001) }, { total: 346063090, datetime: new Date(1362365778292) }, { total: 346063676, datetime: new Date(1362369387496) }, { total: 346102159, datetime: new Date(1362372983589) }, { total: 346106516, datetime: new Date(1362376581997) }, { total: 346108354, datetime: new Date(1362380172428) }, { total: 346135377, datetime: new Date(1362383766845) }, { total: 346140482, datetime: new Date(1362387378939) }, { total: 346156283, datetime: new Date(1362394586447) }, { total: 346160413, datetime: new Date(1362398169435) }, { total: 346173117, datetime: new Date(1362405386582) }, { total: 346176202, datetime: new Date(1362408980407) }, { total: 346188884, datetime: new Date(1362416189380) }, { total: 346191124, datetime: new Date(1362419790815) }, { total: 346191313, datetime: new Date(1362423392031) }, { total: 346203399, datetime: new Date(1362426994065) }, { total: 346203731, datetime: new Date(1362430607995) }, { total: 346203782, datetime: new Date(1362434196104) }, { total: 346224473, datetime: new Date(1362437772275) }, { total: 346227228, datetime: new Date(1362441391493) }, { total: 346227382, datetime: new Date(1362445002461) }, { total: 346249015, datetime: new Date(1362448602163) }, { total: 346252020, datetime: new Date(1362452203186) }, { total: 346252021, datetime: new Date(1362455782426) }, { total: 346283645, datetime: new Date(1362459399052) }, { total: 346306223, datetime: new Date(1362462992079) }, { total: 346334751, datetime: new Date(1362470187140) }, { total: 346363460, datetime: new Date(1362473786489) }, { total: 346386791, datetime: new Date(1362480993367) }, { total: 346395653, datetime: new Date(1362484590510) }, { total: 346408901, datetime: new Date(1362491790792) }, { total: 346413303, datetime: new Date(1362495389507) } ], _id: ObjectId('511e4130a298d8d31600071e') } idhack:1 keyUpdates:1 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:4955 2ms
Tue Mar  5 09:56:29 [conn311367] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x842e39 0x8467c1 0x849227 0x7ae1ef 0x7b4654 0x56c688 0xaf57b5 0x31aa607851 
 /var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
 /var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
 /var/lib/mongo/bin/mongod() [0xacd5cc]
 /var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
 /var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
 /var/lib/mongo/bin/mongod() [0x824e7f]
 /var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
 /var/lib/mongo/bin/mongod() [0x842e39]
 /var/lib/mongo/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x3081) [0x8467c1]
 /var/lib/mongo/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x849227]
 /var/lib/mongo/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0x7ae1ef]
 /var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbd4) [0x7b4654]
 /var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
 /var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
 /lib64/libpthread.so.0() [0x31aa607851]
Tue Mar  5 09:56:29 [conn311367] update fbridge.activity_feed query: { _id: ObjectId('511e3e89a298d8c91300009d') } update: { user_id: 165710, created_at: new Date(1360936585660), updated_at: new Date(1362495389683), network: "twitter", source: "followers", external_id: "221918717", data: { title: "profile stats", importance: 21, total: 4597 }, activities: [ { total: 4452, datetime: new Date(1361010492725) }, { total: 4453, datetime: new Date(1361024889455) }, { total: 4454, datetime: new Date(1361039296622) }, { total: 4457, datetime: new Date(1361046500412) }, { total: 4459, datetime: new Date(1361053693225) }, { total: 4461, datetime: new Date(1361060906789) }, { total: 4462, datetime: new Date(1361075295338) }, { total: 4464, datetime: new Date(1361082486722) }, { total: 4465, datetime: new Date(1361089700348) }, { total: 4466, datetime: new Date(1361096884614) }, { total: 4467, datetime: new Date(1361118504998) }, { total: 4468, datetime: new Date(1361125693487) }, { total: 4469, datetime: new Date(1361140106779) }, { total: 4470, datetime: new Date(1361154504897) }, { total: 4473, datetime: new Date(1361161693609) }, { total: 4476, datetime: new Date(1361168890816) }, { total: 4477, datetime: new Date(1361204897505) }, { total: 4481, datetime: new Date(1361212116607) }, { total: 4482, datetime: new Date(1361219309225) }, { total: 4484, datetime: new Date(1361226511133) }, { total: 4485, datetime: new Date(1361234139730) }, { total: 4484, datetime: new Date(1361241769693) }, { total: 4485, datetime: new Date(1361249377018) }, { total: 4486, datetime: new Date(1361257011427) }, { total: 4487, datetime: new Date(1361279236204) }, { total: 4489, datetime: new Date(1361294516943) }, { total: 4490, datetime: new Date(1361302096782) }, { total: 4492, datetime: new Date(1361324961264) }, { total: 4493, datetime: new Date(1361378134986) }, { total: 4494, datetime: new Date(1361385741808) }, { total: 4495, datetime: new Date(1361400977422) }, { total: 4503, datetime: new Date(1361408424137) }, { total: 4504, datetime: new Date(1361416059541) }, { total: 4507, datetime: new Date(1361423673384) }, { total: 4508, datetime: new Date(1361453712116) }, { total: 4509, datetime: new Date(1361461353632) }, { total: 4510, datetime: new Date(1361468971037) }, { total: 4512, datetime: new Date(1361476593070) }, { total: 4513, datetime: new Date(1361491645504) }, { total: 4517, datetime: new Date(1361499284468) }, { total: 4518, datetime: new Date(1361514514185) }, { total: 4519, datetime: new Date(1361537360835) }, { total: 4520, datetime: new Date(1361544990425) }, { total: 4521, datetime: new Date(1361560068925) }, { total: 4522, datetime: new Date(1361575296421) }, { total: 4523, datetime: new Date(1361582904025) }, { total: 4524, datetime: new Date(1361590546730) }, { total: 4526, datetime: new Date(1361598149470) }, { total: 4527, datetime: new Date(1361605752264) }, { total: 4528, datetime: new Date(1361643709644) }, { total: 4529, datetime: new Date(1361658924566) }, { total: 4528, datetime: new Date(1361666549888) }, { total: 4532, datetime: new Date(1361674191897) }, { total: 4533, datetime: new Date(1361719701036) }, { total: 4532, datetime: new Date(1361734978101) }, { total: 4534, datetime: new Date(1361742635957) }, { total: 4537, datetime: new Date(1361765283703) }, { total: 4538, datetime: new Date(1361772870508) }, { total: 4539, datetime: new Date(1361780486230) }, { total: 4541, datetime: new Date(1361803354597) }, { total: 4546, datetime: new Date(1361818588450) }, { total: 4545, datetime: new Date(1361841298843) }, { total: 4548, datetime: new Date(1361848907518) }, { total: 4547, datetime: new Date(1361864165414) }, { total: 4548, datetime: new Date(1361879375078) }, { total: 4549, datetime: new Date(1361887010507) }, { total: 4553, datetime: new Date(1361894458765) }, { total: 4555, datetime: new Date(1361902055273) }, { total: 4557, datetime: new Date(1361909703566) }, { total: 4558, datetime: new Date(1361940187582) }, { total: 4559, datetime: new Date(1361962845856) }, { total: 4561, datetime: new Date(1361993312405) }, { total: 4562, datetime: new Date(1362000959330) }, { total: 4563, datetime: new Date(1362023800453) }, { total: 4564, datetime: new Date(1362061714245) }, { total: 4565, datetime: new Date(1362069352405) }, { total: 4566, datetime: new Date(1362076955801) }, { total: 4567, datetime: new Date(1362084589182) }, { total: 4568, datetime: new Date(1362099648350) }, { total: 4569, datetime: new Date(1362122504388) }, { total: 4571, datetime: new Date(1362168063232) }, { total: 4572, datetime: new Date(1362183326315) }, { total: 4573, datetime: new Date(1362190922410) }, { total: 4578, datetime: new Date(1362259322264) }, { total: 4579, datetime: new Date(1362266939694) }, { total: 4580, datetime: new Date(1362274536657) }, { total: 4582, datetime: new Date(1362282152885) }, { total: 4585, datetime: new Date(1362289805683) }, { total: 4586, datetime: new Date(1362320476631) }, { total: 4585, datetime: new Date(1362328105123) }, { total: 4586, datetime: new Date(1362335714252) }, { total: 4587, datetime: new Date(1362350961394) }, { total: 4589, datetime: new Date(1362358579732) }, { total: 4592, datetime: new Date(1362366036740) }, { total: 4593, datetime: new Date(1362373627602) }, { total: 4592, datetime: new Date(1362388885446) }, { total: 4593, datetime: new Date(1362434436199) }, { total: 4595, datetime: new Date(1362457286968) }, { total: 4596, datetime: new Date(1362464890616) }, { total: 4597, datetime: new Date(1362495389683) } ], _id: ObjectId('511e3e89a298d8c91300009d') } idhack:1 keyUpdates:1 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:5091 2ms
Tue Mar  5 09:56:29 [conn311369] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x842e39 0x8467c1 0x849227 0x7ae1ef 0x7b4654 0x56c688 0xaf57b5 0x31aa607851 
 /var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
 /var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
 /var/lib/mongo/bin/mongod() [0xacd5cc]
 /var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
 /var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
 /var/lib/mongo/bin/mongod() [0x824e7f]
 /var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
 /var/lib/mongo/bin/mongod() [0x842e39]
 /var/lib/mongo/bin/mongod(_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x3081) [0x8467c1]
 /var/lib/mongo/bin/mongod(_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xb7) [0x849227]
 /var/lib/mongo/bin/mongod(_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x4cf) [0x7ae1ef]
 /var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbd4) [0x7b4654]
 /var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
 /var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
 /lib64/libpthread.so.0() [0x31aa607851]
Tue Mar  5 09:56:29 [conn311369] update fbridge.activity_feed query: { _id: ObjectId('51200078a298d8bf160023a3') } update: { user_id: 187354, created_at: new Date(1361051768998), updated_at: new Date(1362495389898), network: "facebook", source: "likes", external_id: "354824454594731", data: { title: "profile stats", importance: 21, total: 1072 }, activities: [ { total: 1064, datetime: new Date(1361051768998) }, { total: 1065, datetime: new Date(1361068506727) }, { total: 1066, datetime: new Date(1361118918627) }, { total: 1067, datetime: new Date(1361135663429) }, { total: 1069, datetime: new Date(1361186081004) }, { total: 1070, datetime: new Date(1361203005482) }, { total: 1069, datetime: new Date(1361236868640) }, { total: 1068, datetime: new Date(1361287280168) }, { total: 1069, datetime: new Date(1361337708743) }, { total: 1070, datetime: new Date(1361405224516) }, { total: 1071, datetime: new Date(1361472560335) }, { total: 1072, datetime: new Date(1361506051164) }, { total: 1073, datetime: new Date(1361556457661) }, { total: 1074, datetime: new Date(1361573765574) }, { total: 1075, datetime: new Date(1361624185245) }, { total: 1076, datetime: new Date(1361674584233) }, { total: 1077, datetime: new Date(1361725036623) }, { total: 1079, datetime: new Date(1361809267815) }, { total: 1081, datetime: new Date(1361842935425) }, { total: 1082, datetime: new Date(1361859684096) }, { total: 1083, datetime: new Date(1361910101728) }, { total: 1084, datetime: new Date(1361994547900) }, { total: 1079, datetime: new Date(1362011722643) }, { total: 1081, datetime: new Date(1362028875509) }, { total: 1082, datetime: new Date(1362062566107) }, { total: 1079, datetime: new Date(1362079283567) }, { total: 1070, datetime: new Date(1362096090636) }, { total: 1072, datetime: new Date(1362113390600) }, { total: 1074, datetime: new Date(1362148086695) }, { total: 1072, datetime: new Date(1362165253631) }, { total: 1071, datetime: new Date(1362251247893) }, { total: 1072, datetime: new Date(1362268582590) }, { total: 1071, datetime: new Date(1362337255954) }, { total: 1068, datetime: new Date(1362354616010) }, { total: 1069, datetime: new Date(1362389705872) }, { total: 1070, datetime: new Date(1362460042510) }, { total: 1072, datetime: new Date(1362495389898) } ], _id: ObjectId('51200078a298d8bf160023a3') } idhack:1 nmoved:1 keyUpdates:0 exception: Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115 code:10334 locks(micros) w:5151 2ms
Tue Mar  5 09:56:30 [conn339077] Assertion: 10334:Invalid BSONObj size: 1947271168 (0x00001174) first element: : ?type=115
0xb07561 0xacd449 0xacd5cc 0x5721e8 0x796fed 0x874ac4 0x6583e4 0x814697 0x814988 0x861ace 0x824e7f 0x821779 0x7ac5b2 0x7acdd9 0x7b4644 0x56c688 0xaf57b5 0x31aa607851 0x31aa2e811d 
 /var/lib/mongo/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xb07561]
 /var/lib/mongo/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x99) [0xacd449]
 /var/lib/mongo/bin/mongod() [0xacd5cc]
 /var/lib/mongo/bin/mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x468) [0x5721e8]
 /var/lib/mongo/bin/mongod(_ZN5mongo13unindexRecordEPNS_16NamespaceDetailsEPNS_6RecordERKNS_7DiskLocEb+0x11d) [0x796fed]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr12deleteRecordEPKcPNS_6RecordERKNS_7DiskLocEbbb+0x204) [0x874ac4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails11cappedAllocEPKci+0x4b4) [0x6583e4]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails6_allocEPKci+0x27) [0x814697]
 /var/lib/mongo/bin/mongod(_ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE+0x38) [0x814988]
 /var/lib/mongo/bin/mongod(_ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci+0x6e) [0x861ace]
 /var/lib/mongo/bin/mongod() [0x824e7f]
 /var/lib/mongo/bin/mongod(_ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pbb+0x49) [0x821779]
 /var/lib/mongo/bin/mongod(_ZN5mongo14checkAndInsertEPKcRNS_7BSONObjE+0xc2) [0x7ac5b2]
 /var/lib/mongo/bin/mongod(_ZN5mongo14receivedInsertERNS_7MessageERNS_5CurOpE+0x739) [0x7acdd9]
 /var/lib/mongo/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xbc4) [0x7b4644]
 /var/lib/mongo/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x98) [0x56c688]
 /var/lib/mongo/bin/mongod(_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x415) [0xaf57b5]
 /lib64/libpthread.so.0() [0x31aa607851]
 /lib64/libc.so.6(clone+0x6d) [0x31aa2e811d]



 Comments   
Comment by Stennie Steneker (Inactive) [ 28/Oct/13 ]

Hi Jonathan,

I'm closing this issue due to inactivity.

If you are still seeing this issue (particularly with a newer version of MongoDB, such as 2.4.x) please feel free to open a new issue or comment on this one with the relevant details.

Thanks,
Stephen

Comment by sam.helman@10gen.com [ 08/Mar/13 ]

We recommend upgrading the PHP driver as soon as you can, but we should also look into other potential causes for the problem, since we do not see any specific PHP issues that we can be certain are at fault. Are there any logs you can attach from a primary right before the problem started to occur? This could help us determine if the problem is data corruption, in which case running a validate (http://docs.mongodb.org/manual/reference/command/validate/) on the offending collection would be the next step forward.

Comment by sam.helman@10gen.com [ 08/Mar/13 ]

You can check the logging level with

db.getProfilingStatus()

Comment by Jonathan Travin [ 08/Mar/13 ]

The php driver version is: 1.2.10.
As for the loggin level, if you are referring to the profiling, we don't have it enabled (the records I pasted are from the mongo.log directly).
How can I check the logging level?

Thank you!

Comment by sam.helman@10gen.com [ 08/Mar/13 ]

I'm checking in about the PHP driver, but in the meantime, what level of logging do you have on for the database where the issue is occurring? Additionally, what number version (x.x) of the PHP driver are you using?

Comment by Jonathan Travin [ 08/Mar/13 ]

Hi Sam, thanks for checking this. Let me answer your questions:

1. What driver and driver version are you using to connect to MongoDB?
Right now we are using the old php driver (Mongo class). We know it is deprecated, and we are updating it in the next weeks, but not sure if this could be related to this.

2. Do the errors occur on every update once they begin, or only occasionally?
Once this error start, it occurs for every update, in every collection of the database (we are using only one database). Not sure if it's helpful, but the data it's written correctly in the collection, but still the error is logged.

3. Does your usage pattern include a lot of $pushes and $pulls to embedded arrays within documents?
Not really, most of them are simple updates on a field, some of them are updates on the array (php), and then of course saving it to mongo. Not sure if internally the Mongo class interprets that as $push or it rewrites the whole document.

Thanks!

Comment by sam.helman@10gen.com [ 08/Mar/13 ]

Hello,

We have a few questions that should help us with solving your issue.

1. What driver and driver version are you using to connect to MongoDB?
2. Do the errors occur on every update once they begin, or only occasionally?
3. Does your usage pattern include a lot of $pushes and $pulls to embedded arrays within documents?

Thank you

Generated at Thu Feb 08 03:18:38 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.