Loading...

Type: Bug
Resolution: Unresolved
Priority: Critical - P2
Fix Version/s: None
Affects Version/s: None
Component/s: Disagg CI-blocker, Test Format
Labels:
- dc

Assigned Teams:

Storage Engines, Storage Engines - Persistence
Sprint:
SE Persistence backlog
Story Points:
None

Running test/format with a custom config (attached in the comments) consistently fails due to out-of-order key returns.

[1755405648:701458][22797:0xe7a44340f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405649:470128][22797:0xe7a473e0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405649:470742][22797:0xe7a44d60f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405649:790434][22797:0xe7a46600f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405650:015600][22797:0xe7a46740f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405650:041116][22797:0xe7a44e00f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405650:114531][22797:0xe7a442a0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405651:183142][22797:0xe7a45960f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405651:588898][22797:0xe7a45aa0f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405653:210103][22797:0xe7a464c0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405655:836921][22797:0xe7a458c0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405657:165264][22797:0xe7a45b40f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405657:282798][22797:0xe7a47160f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405657:671373][22797:0xe7a44fe0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405657:895548][22797:0xe7a443e0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405657:944183][22797:0xe7a44ea0f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405659:009593][22797:0xe7a44cc0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405659:271943][22797:0xe7a467e0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405659:353407][22797:0xe7a45a00f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405659:364850][22797:0xe7a470c0f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405659:595425][22797:0xe7a47340f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405660:081720][22797:0xe7a45be0f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405660:977653][22797:0xe7a47200f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405661:747943][22797:0xe7a472a0f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405661:903515][22797:0xe7a44200f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405661:963964][22797:0xe7a466a0f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405662:077507][22797:0xe7a44f40f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405662:155271][22797:0xe7a46560f120], t, file:wt.wt_stable, WT_CURSOR.prev: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.prev out-of-order returns: returned key 0000520026.00/opqrstuvwxyzabcdefghi then key 00008073LMNOPQRSTUVWXYZABCDEFG
[1755405663:275588][22797:0xe7a44160f120], t, file:wt.wt_stable, WT_CURSOR.next: [WT_VERB_OUT_OF_ORDER][ERROR]: WT_CURSOR.next out-of-order returns: returned key 00008073vwxyzabcdefgC0000807368/P then key 00008073opqrstuvwxyzabcdefghijklmnopq70000807
[1755405664:366120][22797:0xe7a43740f120], t, file:wt.wt_stable, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_cell_unpack_kv, 1438: WiredTiger assertion failed: 'ret == 0'. Expression returned false
[1755405664:366149][22797:0xe7a43740f120], t, file:wt.wt_stable, WT_SESSION.checkpoint: [WT_VERB_DEFAULT][ERROR]: __wt_abort, 29: aborting WiredTiger library

related to

WT-15142 Disagg test/format disagg.mode=leader testing triage

In Progress

Details

Description

Attachments

Issue Links

Forms

Activity

People

Dates