Loading...

Type: Bug
Resolution: Done
Priority: Major - P3
Fix Version/s: None
Affects Version/s: 2.6.8
Component/s: Internal Code
Labels:
None

Operating System:
ALL
Steps To Reproduce:

Hide

Conditionally Randomly Reoccure

How do I remove the valid data?

Show
Conditionally Randomly Reoccure How do I remove the valid data?
Confidence Status:
None
Work Order:
3

Aha! Reference:
None
Tracking Level:
None
Risk Status:
None
Exec Notes:
None
Goal Name:
None
Goal Link:
None

version: 2.6.8
3 nodes replica set
--------------- mongodb.log

2015-06-19T23:35:22.445+0800 [conn2313] ceilometer.meter 0xffec79 0xfa7ea5 0xf90716 0xf9084c 0x7dc8a7 0xdd8dfd 0xdd99d9 0xdc0fd9 0xdc1435 0xdc14ce 0xdc14ce 0xdc14ce 0xdc164f 0xb00d13 0xaf9014 0x8f756d 0x8dfa6a 0x8e037d 0x9e023e 0x9e0c09 
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xffec79]
 mongod(_ZN5mongo10logContextEPKc+0x1f5) [0xfa7ea5]
 mongod(_ZN5mongo11msgassertedEiPKc+0xe6) [0xf90716]
 mongod() [0xf9084c]
 mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x3a7) [0x7dc8a7]
 mongod(_ZNK5mongo5KeyV16toBsonEv+0xa3d) [0xdd8dfd]
 mongod(_ZNK5mongo5KeyV113compareHybridERKS0_RKNS_8OrderingE+0x39) [0xdd99d9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE4findEPKNS_17IndexCatalogEntryERKNS_5KeyV1ERKNS_7DiskLocERib+0x109) [0xdc0fd9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0x55) [0xdc1435]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_7BSONObjERiRbS8_i+0x7f) [0xdc164f]
 mongod(_ZNK5mongo18BtreeInterfaceImplINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_7BSONObjERiRbS8_i+0x73) [0xb00d13]
 mongod(_ZN5mongo22BtreeBasedAccessMethod5touchERKSt3setINS_7BSONObjENS_10BSONObjCmpESaIS2_EE+0xa4) [0xaf9014]
 mongod(_ZNK5mongo12IndexCatalog5touchEPKNS_16PregeneratedKeysE+0xed) [0x8f756d]
 mongod(_ZN5mongo10Collection15_insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x4a) [0x8dfa6a]
 mongod(_ZN5mongo10Collection14insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x1ad) [0x8e037d]
 mongod(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0x21e) [0x9e023e]
 mongod(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x399) [0x9e0c09]
2015-06-19T23:35:22.445+0800 [conn2296] Assertion: 10334:BSONObj size: -2 (0xFFFFFFFE) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO
2015-06-19T23:35:22.445+0800 [conn2313] end connection 172.28.0.4:47945 (97 connections now open)
2015-06-19T23:35:22.455+0800 [conn2296] ceilometer.meter 0xffec79 0xfa7ea5 0xf90716 0xf9084c 0x7dc8a7 0xdd8dfd 0xdd99d9 0xdc0fd9 0xdc1435 0xdc14ce 0xdc14ce 0xdc14ce 0xdc164f 0xb00d13 0xaf9014 0x8f756d 0x8dfa6a 0x8e037d 0x9e023e 0x9e0c09 
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xffec79]
 mongod(_ZN5mongo10logContextEPKc+0x1f5) [0xfa7ea5]
 mongod(_ZN5mongo11msgassertedEiPKc+0xe6) [0xf90716]
 mongod() [0xf9084c]
 mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x3a7) [0x7dc8a7]
 mongod(_ZNK5mongo5KeyV16toBsonEv+0xa3d) [0xdd8dfd]
 mongod(_ZNK5mongo5KeyV113compareHybridERKS0_RKNS_8OrderingE+0x39) [0xdd99d9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE4findEPKNS_17IndexCatalogEntryERKNS_5KeyV1ERKNS_7DiskLocERib+0x109) [0xdc0fd9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0x55) [0xdc1435]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_7BSONObjERiRbS8_i+0x7f) [0xdc164f]
 mongod(_ZNK5mongo18BtreeInterfaceImplINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_7BSONObjERiRbS8_i+0x73) [0xb00d13]
 mongod(_ZN5mongo22BtreeBasedAccessMethod5touchERKSt3setINS_7BSONObjENS_10BSONObjCmpESaIS2_EE+0xa4) [0xaf9014]
 mongod(_ZNK5mongo12IndexCatalog5touchEPKNS_16PregeneratedKeysE+0xed) [0x8f756d]
 mongod(_ZN5mongo10Collection15_insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x4a) [0x8dfa6a]
 mongod(_ZN5mongo10Collection14insertDocumentERKNS_7BSONObjEbPKNS_16PregeneratedKeysE+0x1ad) [0x8e037d]
 mongod(_ZN5mongo18WriteBatchExecutor13execOneInsertEPNS0_16ExecInsertsStateEPPNS_16WriteErrorDetailE+0x21e) [0x9e023e]
 mongod(_ZN5mongo18WriteBatchExecutor11execInsertsERKNS_21BatchedCommandRequestEPSt6vectorIPNS_16WriteErrorDetailESaIS6_EE+0x399) [0x9e0c09]
2015-06-19T23:35:22.455+0800 [conn2296] insert ceilometer.meter query: { _id: ObjectId('558436ba97f6435eb47a0ba5'), counter_name: "quota.ram", user_id: null, message_signature: "f92ed97306f2b348b577236d49aaf682e539bfce361b2be0024ec670146d1d99", timestamp: new Date(1434725710000), resource_id: "398408b47f1e4bc4a5b50c0447f1e3d1", resource_metadata: { limitCores: 20480, limitRam: 25165824, totalCoresUsed: 0, totalRAMUsed: 0, limitInstances: 2048, totalInstancesUsed: 0, domain_id: "default", additionalInfo: "" }, source: "openstack", counter_unit: "%", counter_volume: 0.0, recorded_at: new Date(1434728122346), project_id: "398408b47f1e4bc4a5b50c0447f1e3d1", message_id: "2eaceaa4-1693-11e5-867f-9017acae562f", counter_type: "gauge" } ninserted:0 keyUpdates:0 exception: BSONObj size: -2 (0xFFFFFFFE) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO code:10334 numYields:0 locks(micros) w:9864 108ms
2015-06-19T23:35:22.455+0800 [conn2301] Assertion: 10334:BSONObj size: -2 (0xFFFFFFFE) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO
2015-06-19T23:35:22.455+0800 [conn2296] command ceilometer.$cmd command: insert { insert: "meter", ordered: true, documents: [ { _id: ObjectId('558436ba97f6435eb47a0ba5'), counter_name: "quota.ram", user_id: null, message_signature: "f92ed97306f2b348b577236d49aaf682e539bfce361b2be0024ec670146d1d99", timestamp: new Date(1434725710000), resource_id: "398408b47f1e4bc4a5b50c0447f1e3d1", resource_metadata: { limitCores: 20480, limitRam: 25165824, totalCoresUsed: 0, totalRAMUsed: 0, limitInstances: 2048, totalInstancesUsed: 0, domain_id: "default", additionalInfo: "" }, source: "openstack", counter_unit: "%", counter_volume: 0.0, recorded_at: new Date(1434728122346), project_id: "398408b47f1e4bc4a5b50c0447f1e3d1", message_id: "2eaceaa4-1693-11e5-867f-9017acae562f", counter_type: "gauge" } ] } keyUpdates:0 numYields:0  reslen:242 108ms
2015-06-19T23:35:22.465+0800 [conn2296] end connection 172.28.0.4:47922 (96 connections now open)
2015-06-19T23:35:22.465+0800 [conn2301] ceilometer.meter 0xffec79 0xfa7ea5 0xf90716 0xf9084c 0x7dc8a7 0xdd8dfd 0xdd99d9 0xdc0fd9 0xdc1435 0xdc14ce 0xdc14ce 0xdc14ce 0xdc164f 0xb00d13 0xaf9014 0x8f756d 0x8dfa6a 0x8e037d 0x9e023e 0x9e0c09 
 mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xffec79]
 mongod(_ZN5mongo10logContextEPKc+0x1f5) [0xfa7ea5]
 mongod(_ZN5mongo11msgassertedEiPKc+0xe6) [0xf90716]
 mongod() [0xf9084c]
 mongod(_ZNK5mongo7BSONObj14_assertInvalidEv+0x3a7) [0x7dc8a7]
 mongod(_ZNK5mongo5KeyV16toBsonEv+0xa3d) [0xdd8dfd]
 mongod(_ZNK5mongo5KeyV113compareHybridERKS0_RKNS_8OrderingE+0x39) [0xdd99d9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE4findEPKNS_17IndexCatalogEntryERKNS_5KeyV1ERKNS_7DiskLocERib+0x109) [0xdc0fd9]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0x55) [0xdc1435]
 mongod(_ZNK5mongo11BtreeBucketINS_12BtreeData_V1EE6locateEPKNS_17IndexCatalogEntryERKNS_7DiskLocERKNS_5KeyV1ERiRbS8_i+0xee) [0xdc14ce]

Details

Description

Attachments

Activity

People

Dates