[SERVER-48637] error:WiredTiger error (22) and Fatal assertion 39001 Created: 06/Jun/20 Updated: 29/Sep/21 Resolved: 12/Apr/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 4.2.7 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | kkkkkk kopone | Assignee: | Dmitry Agranat |
| Resolution: | Duplicate | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Description |
|
| Comments |
| Comment by Dmitry Agranat [ 12/Apr/21 ] |
|
Thanks srikanta@jungleegames.com, we have moved |
| Comment by Srikanta Patra [ 08/Apr/21 ] |
|
@Dmitry Agranat, I create separete ticket. Please find below link ticket. https://jira.mongodb.org/browse/WT-7398
|
| Comment by Srikanta Patra [ 08/Apr/21 ] |
|
@Dmitry Agranat, Sure I will create ticket and sharing to you. |
| Comment by Dmitry Agranat [ 06/Apr/21 ] |
|
2857516870@qq.com, We believe the issue you've originally reported is related to SERVER-55746, please follow SERVER-55746 for further updates. zubair.iub.pk@gmail.com, We are currently investigating the issue you've reported in srikanta@jungleegames.com, divomen@gmail.com If this is still an issue for you, please open a separate ticket where we'll need to collect some diagnostics data in order to investigate it. |
| Comment by Srikanta Patra [ 05/Apr/21 ] |
|
Hi All,
We are also getting same error on production server. Please help to get it resolve. 2021-04-02T22:52:49.692+0530 E STORAGE [conn4020] WiredTiger error (22) [1617384169:692561][5626:0x7f620f4f5700], WT_SESSION.timestamp_transaction: __wt_txn_set_commit_timestamp, 683: commit timestamp (1617384169, 810) is less than the oldest timestamp (1617384169, 811): Invalid argument Raw: [1617384169:692561][5626:0x7f620f4f5700], WT_SESSION.timestamp_transaction: __wt_txn_set_commit_timestamp, 683: commit timestamp (1617384169, 810) is less than the oldest timestamp (1617384169, 811): Invalid argument |
| Comment by Zubair Ahmed [ 03/Apr/21 ] |
|
Neither manually nor inserted documents directly into oplog of replica set. |
| Comment by Daniel Gottlieb (Inactive) [ 02/Apr/21 ] |
|
zubair.iub.pk@gmail.com are you or an application being run inserting documents directly into the oplog of an active replica set primary? |
| Comment by Zubair Ahmed [ 02/Apr/21 ] |
|
Mongodb Version : 4.2.5 service crashed on Centos 7.7.1908 with following backtrace
2021-03-31T13:32:03.820+0000 F - [conn527851316] Got signal: 6 (Aborted). ,{"b":"56454D5D4000","o":"28A973E"},{"b":"56454D5D4000","o":"28A97D6"},{"b":"7FAA798D9000","o":"F5F0"},{"b":"7FAA7950B000","o":"36337","s":"gsignal"},{"b":"7FAA7950B000","o":"37A28","s":"abort"},{"b":"56454D5D4000","o":"D0769D","s":"_ZN5mongo42fassertFailedWithStatusNoTraceWithLocationEiRKNS_6StatusEPKcj"},{"b":"56454D5D4000","o":"A46395"},{"b":"56454D5D4000","o":"E040FC","s":"_ZN5mongo21WiredTigerRecordStore26insertRecordsWithDocWriterEPNS_16OperationContextEPKPKNS_9DocWriterEPKNS_9TimestampEmPNS_8RecordIdE"},{"b":"56454D5D4000","o":"15D8071","s":"_ZN5mongo14CollectionImpl23insertDocumentsForOplogEPNS_16OperationContextEPKPKNS_9DocWriterEPNS_9TimestampEm"},{"b":"56454D5D4000","o":"166B4E3","s":"_ZN5mongo4repl12_logOpsInnerEPNS_16OperationContextERKNS_15NamespaceStringEPKPKNS_9DocWriterEPNS_9TimestampEmPNS_10CollectionENS0_6OpTimeENS_6Date_tE"},{"b":"56454D5D4000","o":"166FC7C","s":"_ZN5mongo4repl5logOpEPNS_16OperationContextEPKcRKNS_15NamespaceStringEN5boost8optionalINS_4UUIDEEERKNS_7BSONObjEPSD_bNS_6Date_tERKNS_20OperationSessionInfoENS9_IiEERKNS0_9OplogLinkERKNS0_6OpTimeE"},{"b":"56454D5D4000","o":"14563CB"},{"b":"56454D5D4000","o":"145C3CC","s":"_ZN5mongo14OpObserverImpl8onUpdateEPNS_16OperationContextERKNS_20OplogUpdateEntryArgsE"},{"b":"56454D5D4000","o":"D91DC7","s":"_ZN5mongo18OpObserverRegistry8onUpdateEPNS_16OperationContextERKNS_20OplogUpdateEntryArgsE"},{"b":"56454D5D4000","o":"15DDE4B","s":"_ZN5mongo14CollectionImpl14updateDocumentEPNS_16OperationContextENS_8RecordIdERKNS_11SnapshottedINS_7BSONObjEEERKS5_bPNS_7OpDebugEPNS_20CollectionUpdateArgsE"},{"b":"56454D5D4000","o":"179AF6D","s":"_ZN5mongo11UpdateStage18transformAndUpdateERKNS_11SnapshottedINS_7BSONObjEEERNS_8RecordIdE"},{"b":"56454D5D4000","o":"179BE12","s":"_ZN5mongo11UpdateStage6doWorkEPm"},{"b":"56454D5D4000","o":"177A518","s":"_ZN5mongo9PlanStage4workEPm"},{"b":"56454D5D4000","o":"177A897","s":"_ZN5mongo15ProjectionStage6doWorkEPm"},{"b":"56454D5D4000","o":"177A518","s":"_ZN5mongo9PlanStage4workEPm"},{"b":"56454D5D4000","o":"17C1850","s":"_ZN5mongo16PlanExecutorImpl12_getNextImplEPNS_11SnapshottedINS_7BSONObjEEEPNS_8RecordIdE"},{"b":"56454D5D4000","o":"17C1FAD","s":"_ZN5mongo16PlanExecutorImpl7getNextEPNS_7BSONObjEPNS_8RecordIdE"},{"b":"56454D5D4000","o":"1487617"},{"b":"56454D5D4000","o":"1489410"},{"b":"56454D5D4000","o":"148A66A"},{"b":"56454D5D4000","o":"229C67F","s":"_ZN5mongo12BasicCommand10Invocation3runEPNS_16OperationContextEPNS_3rpc21ReplyBuilderInterfaceE"},{"b":"56454D5D4000","o":"11A8EAA"},{"b":"56454D5D4000","o":"11AA1B2"},{"b":"56454D5D4000","o":"11AC204"},{"b":"56454D5D4000","o":"11ACF6A","s":"_ZN5mongo23ServiceEntryPointCommon13handleRequestEPNS_16OperationContextERKNS_7MessageERKNS0_5HooksE"},{"b":"56454D5D4000","o":"119ABBC","s":"_ZN5mongo23ServiceEntryPointMongod13handleRequestEPNS_16OperationContextERKNS_7MessageE"},{"b":"56454D5D4000","o":"11A694C","s":"_ZN5mongo19ServiceStateMachine15_processMessageENS0_11ThreadGuardE"},{"b":"56454D5D4000","o":"11A3C2F","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56454D5D4000","o":"11A58BC"},{"b":"56454D5D4000","o":"1FD7572","s":"_ZN5mongo9transport26ServiceExecutorSynchronous8scheduleESt8functionIFvvEENS0_15ServiceExecutor13ScheduleFlagsENS0_23ServiceExecutorTaskNameE"},{"b":"56454D5D4000","o":"11A0AFD","s":"_ZN5mongo19ServiceStateMachine22_scheduleNextWithGuardENS0_11ThreadGuardENS_9transport15ServiceExecutor13ScheduleFlagsENS2_23ServiceExecutorTaskNameENS0_9OwnershipE"},{"b":"56454D5D4000","o":"11A2373","s":"_ZN5mongo19ServiceStateMachine15_sourceCallbackENS_6StatusE"},{"b":"56454D5D4000","o":"11A2F26","s":"_ZN5mongo19ServiceStateMachine14_sourceMessageENS0_11ThreadGuardE"},{"b":"56454D5D4000","o":"11A3B8B","s":"_ZN5mongo19ServiceStateMachine15_runNextInGuardENS0_11ThreadGuardE"},{"b":"56454D5D4000","o":"11A58BC"},{"b":"56454D5D4000","o":"1FD79DB"},{"b":"56454D5D4000","o":"2626F64"},{"b":"7FAA798D9000","o":"7E65"},{"b":"7FAA7950B000","o":"FE88D","s":"clone"}],"processInfo":{ "mongodbVersion" : "4.2.5", "gitVersion" : "2261279b51ea13df08ae708ff278f0679c59dc32", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "3.10.0-1062.9.1.el7.x86_64", "version" : "#1 SMP Fri Dec 6 15:49:49 UTC 2019", "machine" : "x86_64" }, "somap" : [ { "b" : "56454D5D4000", "elfType" : 3, "buildId" : "56B1321DBA5855D67E2937EDEADAE0C9D69BBBA4" }, { "b" : "7FFCE9CA6000", "elfType" : 3, "buildId" : "B5A5458535A1397FA6BAAF5E8C13A6395426A1B2" }, { "b" : "7FAA7AD07000", "path" : "/lib64/libcurl.so.4", "elfType" : 3, "buildId" : "89C83CEB5DE5FDEC4F3DFBA4FCAACB53D747A998" }, { "b" : "7FAA7AAEE000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "3009B26B33156EAAF99787AA3DA0C6AE99649755" }, { "b" : "7FAA7A68B000", "path" : "/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "4CF1939F660008CFA869D8364651F31AACD2C1C4" }, { "b" : "7FAA7A419000", "path" : "/lib64/libssl.so.10", "elfType" : 3, "buildId" : "3B305C3BA17FE394862E749763F2956C9C890C2E" }, { "b" : "7FAA7A215000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "18113E6E83D8E981B8E8D808F7F3DBB23F950A1D" }, { "b" : "7FAA7A00D000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "4749697BF078337576C4629F0D30B296A0939779" }, { "b" : "7FAA79D0B000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "5681C054FDABCF789F4DDA66E94F1F6ED1747327" }, { "b" : "7FAA79AF5000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "DAC0179F4555AEFEC9E97476201802FD20C03EC5" }, { "b" : "7FAA798D9000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "8B33F7F8C86F8D544C63C5541A8E42B3DDFEF8B1" }, { "b" : "7FAA7950B000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "398944D32CF16A67AF51067A326E6C0CC14F90ED" }, { "b" : "7FAA7AF71000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "5CC1A53B747A7E4D21198723C2B633E54F3C06D9" }, { "b" : "7FAA792D8000", "path" : "/lib64/libidn.so.11", "elfType" : 3, "buildId" : "2B77BBEFFF65E94F3E0B71A4E89BEB68C4B476C5" }, { "b" : "7FAA790AB000", "path" : "/lib64/libssh2.so.1", "elfType" : 3, "buildId" : "1AF123CADB2F2910E89CBD540A06D3B33692F95E" }, { "b" : "7FAA78E52000", "path" : "/lib64/libssl3.so", "elfType" : 3, "buildId" : "B6321C434B5C7386B144B925CEE2798D269FDDF5" }, { "b" : "7FAA78C2A000", "path" : "/lib64/libsmime3.so", "elfType" : 3, "buildId" : "BDA454441F59F41D2DA36E13CEA1FC4CE95B2BBB" }, { "b" : "7FAA788FB000", "path" : "/lib64/libnss3.so", "elfType" : 3, "buildId" : "D61EB90C9F32CA6E81E7FAC437F2C496438C8D9E" }, { "b" : "7FAA786CB000", "path" : "/lib64/libnssutil3.so", "elfType" : 3, "buildId" : "1E366A2153AD7488EE72E989D9AD6BD458BE8EDE" }, { "b" : "7FAA784C7000", "path" : "/lib64/libplds4.so", "elfType" : 3, "buildId" : "325B8CE57A776DE0B24B362A7E0C90E903B1A4B8" }, { "b" : "7FAA782C2000", "path" : "/lib64/libplc4.so", "elfType" : 3, "buildId" : "0460FF10A3C63749113D380C40E10DFCF066C76E" }, { "b" : "7FAA78084000", "path" : "/lib64/libnspr4.so", "elfType" : 3, "buildId" : "8840B019EDB66B0CFBD2F77EF196440F7928106E" }, { "b" : "7FAA77E37000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "E2AA8CA3D3164E7DBEC293BFA0B55D2B10DAC05D" }, { "b" : "7FAA77B4E000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "3EE7267AF7BFD3B132E6A222D997DA09C96C90DD" }, { "b" : "7FAA7791B000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "82E28CACB60C27CD6F14A6D2268F0CFF621664D0" }, { "b" : "7FAA77717000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "67E935BFABA2C914C01156B88947DD515EA51170" }, { "b" : "7FAA77508000", "path" : "/lib64/liblber-2.4.so.2", "elfType" : 3, "buildId" : "3192C56CD451E18EB9F29CB045432BA9C738DD29" }, { "b" : "7FAA772B3000", "path" : "/lib64/libldap-2.4.so.2", "elfType" : 3, "buildId" : "F1FADDDE0D21D5F4E2DCADEDD3B85B6E7AAC9883" }, { "b" : "7FAA7709D000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "B9D5F73428BD6AD68C96986B57BEA3B7CEDB9745" }, { "b" : "7FAA76E8D000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "4F5FBB2087BE132892467C4E7A46A3D07E5DA40B" }, { "b" : "7FAA76C89000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "2E01D5AC08C1280D013AAB96B292AC58BC30A263" }, { "b" : "7FAA76A6C000", "path" : "/lib64/libsasl2.so.3", "elfType" : 3, "buildId" : "E2F2017F821DD1B9D307DA1A9B8014F2941AEB7B" }, { "b" : "7FAA76845000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "D2DD4DA3FDE1477D25BFFF80F3A25FDB541A8179" }, { "b" : "7FAA7660E000", "path" : "/lib64/libcrypt.so.1", "elfType" : 3, "buildId" : "84467C988F41D853C58353BEB247670E15DA8BAD" }, { "b" : "7FAA763AC000", "path" : "/lib64/libpcre.so.1", "elfType" : 3, "buildId" : "9CA3D11F018BEEB719CDB34BE800BF1641350D0A" }, { "b" : "7FAA761A9000", "path" : "/lib64/libfreebl3.so", "elfType" : 3, "buildId" : "197680DAE6538245CB99723E57447C4EF2E98362" } ] }} |
| Comment by Dmitry Voronenkov [ 15/Jun/20 ] |
|
I have the same issue: 2020-06-15T00:57:16.354+0000 E STORAGE [conn23415] WiredTiger error (22) [1592182636:354084][1:0x7fbc81a77700], WT_SESSION.timestamp_transaction: __wt_txn_set_commit_timestamp, 683: commit timestamp (1592182636, 108) is less than the oldest timestamp (1592182636, 110): Invalid argument Raw: [1592182636:354084][1:0x7fbc81a77700], WT_SESSION.timestamp_transaction: __wt_txn_set_commit_timestamp, 683: commit timestamp (1592182636, 108) is less than the oldest timestamp (1592182636, 110): Invalid argument ***aborting after fassert() failure 2020-06-15T00:57:16.754+0000 F - [conn23415] Got signal: 6 (Aborted). |
| Comment by Carl Champain (Inactive) [ 08/Jun/20 ] |
|
Thanks for the report. Kind regards, |