-
Type: Improvement
-
Resolution: Duplicate
-
Priority: Minor - P4
-
None
-
Affects Version/s: 3.0.5
-
Component/s: Logging
-
Storage Execution
-
(copied to CRM)
We are experiencing an error where we can find some records with a query on the secondary but not on the primary. We are still working on a reproducible test case for this and will file a bug once we have once.
However.
When a record can't be found the only indication we have in mongodb.log is an assertion log with severity I(nformational):
2015-10-29T01:21:45.869Z I - [conn225391] Assertion: 28556:Didn't find RecordId in WiredTigerRecordStore with an associated stack trace: 2015-10-29T01:21:45.877Z I CONTROL [conn225391] 0xf77ca9 0xf16b61 0xefbdbf 0xd8ff0c 0x92cb70 0xa1cb0b 0xa2dcd4 0xa2e5c7 0xbeaae2 0xbeb14c 0xbeb47f 0xbbccff 0xbb8736 0xaceb60 0x829dcd 0xf2ac4b 0x7fde2d045df5 0x7fde2baf8bfd ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"400000","o":"B77CA9"},{"b":"400000","o":"B16B61"},{"b":"400000","o":"AFBDBF"},{"b":"400000","o":"98FF0C"},{"b":"400000","o":"52CB70"},{"b":"400000","o":"61CB0B"},{"b":"400000","o":"62D CD4"},{"b":"400000","o":"62E5C7"},{"b":"400000","o":"7EAAE2"},{"b":"400000","o":"7EB14C"},{"b":"400000","o":"7EB47F"},{"b":"400000","o":"7BCCFF"},{"b":"400000","o":"7B8736"},{"b":"400000","o":"6CEB60"},{" b":"400000","o":"429DCD"},{"b":"400000","o":"B2AC4B"},{"b":"7FDE2D03E000","o":"7DF5"},{"b":"7FDE2BA02000","o":"F6BFD"}],"processInfo":{ "mongodbVersion" : "3.0.5", "gitVersion" : "8bc4ae20708dbb493cb09338 d9e7be6698e4a3a3", "uname" : { "sysname" : "Linux", "release" : "4.1.7-15.23.amzn1.x86_64", "version" : "#1 SMP Mon Sep 14 23:20:33 UTC 2015", "machine" : "x86_64" }, "somap" : [ { "elfType" : 2, "b" : "4 00000", "buildId" : "E8A046532FE2C5A8474250BA8E5B72C6C84B9D68" }, { "b" : "7FFF5D5FC000", "elfType" : 3, "buildId" : "AF79256455DAB4D8F13725CDA5E71975F69FED3F" }, { "b" : "7FDE2D03E000", "path" : "/lib64/ libpthread.so.0", "elfType" : 3, "buildId" : "21054340E69FD5F7DDC9DECEF7951CF3F3D2E78E" }, { "b" : "7FDE2CDD1000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "22480480235F3B1C6C2E5E595 3949728676D3796" }, { "b" : "7FDE2C9EC000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "F1C146B78505646930DD9003AA2B3622C5226D1B" }, { "b" : "7FDE2C7E4000", "path" : "/lib64/librt.so.1" , "elfType" : 3, "buildId" : "8A6EB7521ABE0518C2803504B52F18412D69C713" }, { "b" : "7FDE2C5E0000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "7030B651A4F5D6F32A8D83F3541990E342C50E6C" }, { "b" : "7FDE2C2DC000", "path" : "/usr/lib64/libstdc++.so.6", "elfType" : 3, "buildId" : "DD6383EEAC49E9BAA9E3D1080AE932F42CF8A385" }, { "b" : "7FDE2BFDA000", "path" : "/lib64/libm.so.6", "elfType" : 3, "bu ildId" : "EB09196C8A859427FD964381CDAC80EC0C4C4296" }, { "b" : "7FDE2BDC4000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "C52958E393BDF8E8D090F36DE0F4E620D8736FBF" }, { "b" : "7FDE2BA020 00", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "0EB7F7FA93C9C3E017F7257AE97415730345767B" }, { "b" : "7FDE2D25A000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "F451C 8F279B6860B00F835764FFBA0D54879E1C6" }, { "b" : "7FDE2B7B6000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "FF843C37C38E5BFFD57F7BCCAE05FDADC6390C8F" }, { "b" : "7FDE2B4D3000", "pat h" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "0BB150CC29DB5B0E039879EFC00152A75E3B00B9" }, { "b" : "7FDE2B2D0000", "path" : "/usr/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "5C01209C5AE1 B1714F19B07EB58F2A1274B69DC8" }, { "b" : "7FDE2B09E000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "1485992B0E5CDBA0A34817FC8C6A4C45E82CD1A9" }, { "b" : "7FDE2AE88000", "path" : "/lib 64/libz.so.1", "elfType" : 3, "buildId" : "89C6AF118B6B4FB6A73AE1813E2C8BDD722956D1" }, { "b" : "7FDE2AC79000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "A75A81EC50E9E0164A12B59D9 987AD61AC7576C8" }, { "b" : "7FDE2AA76000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "37A58210FA50C91E09387765408A92909468D25B" }, { "b" : "7FDE2A85C000", "path" : "/lib64/libresolv. so.2", "elfType" : 3, "buildId" : "F586C9A0950DFD60125B2F56B0EC6D690128B1FC" }, { "b" : "7FDE2A63B000", "path" : "/usr/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "C16D759FEE22609796EA66A648C297C1E B9191D1" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x29) [0xf77ca9] mongod(_ZN5mongo10logContextEPKc+0xE1) [0xf16b61] mongod(_ZN5mongo11msgassertedEiPKc+0xAF) [0xefbdbf] mongod(_ZNK5mongo21WiredTigerRecordStore7dataForEPNS_16OperationContextERKNS_8RecordIdE+0xDC) [0xd8ff0c] mongod(_ZNK5mongo10Collection6docForEPNS_16OperationContextERKNS_8RecordIdE+0x20) [0x92cb70] mongod(_ZN5mongo10FetchStage4workEPm+0x2BB) [0xa1cb0b] mongod(_ZN5mongo14MultiPlanStage12workAllPlansEmPNS_15PlanYieldPolicyE+0xE4) [0xa2dcd4] mongod(_ZN5mongo14MultiPlanStage12pickBestPlanEPNS_15PlanYieldPolicyE+0xC7) [0xa2e5c7] mongod(_ZN5mongo12PlanExecutor12pickBestPlanENS0_11YieldPolicyE+0x72) [0xbeaae2] mongod(_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextEPNS_10WorkingSetEPNS_9PlanStageEPNS_13QuerySolutionEPNS_14CanonicalQueryEPKNS_10CollectionERKSsNS0_11YieldPolicyEPPS0_+0x7C) [0xbeb14c] mongod(_ZN5mongo12PlanExecutor4makeEPNS_16OperationContextEPNS_10WorkingSetEPNS_9PlanStageEPNS_13QuerySolutionEPNS_14CanonicalQueryEPKNS_10CollectionENS0_11YieldPolicyEPPS0_+0x7F) [0xbeb47f] mongod(_ZN5mongo11getExecutorEPNS_16OperationContextEPNS_10CollectionEPNS_14CanonicalQueryENS_12PlanExecutor11YieldPolicyEPPS6_m+0xCF) [0xbbccff] mongod(_ZN5mongo8runQueryEPNS_16OperationContextERNS_7MessageERNS_12QueryMessageERKNS_15NamespaceStringERNS_5CurOpES3_+0x666) [0xbb8736] mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xB10) [0xaceb60] mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xDD) [0x829dcd] mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x34B) [0xf2ac4b] libpthread.so.0(+0x7DF5) [0x7fde2d045df5] libc.so.6(clone+0x6D) [0x7fde2baf8bfd] ----- END BACKTRACE -----
There were no messages with Warning or Error severity to report the failure to find the record. This failure can lead to loss of data, and warrants higher severity, and probably a dedicated error message.
- is related to
-
SERVER-22457 Assertion: 28556:Didn't find RecordId in WiredTigerRecordStore
- Closed
-
SERVER-27038 assertion messages should have correct severities
- Closed