Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12077

Incorrect hardware checksum calculation on zSeries for buffers on stack

    • 8
    • 2024-01-09 - I Grew Tired
    • v7.2, v7.0, v6.0, v5.0

      I found this issue while working on WT-11899, when I added additional testing over 0xFF pattern in test/csuite/wt2695_checksum. I only reproduce this issue when testing manually and not on evergreen zSeries hosts. I will add more details in the comments on how to reproduce. The summary is that after my test changes, wt2695_checksum allocates a buffer initialised to 0xFF on the stack. The CRC calculation on that buffer fails if an intermediate pointer to the array is used as the argument to the CRC calculation function. The calculation starts to pass again on introducing more allocations on the stack.

      I suspect that the internal function we use on zSeries, __wt_crc32c_le_vx, is susceptible to alignments. Though the readme says that the testing has been done across various alignments for the hardware routine. The routine we use is a combination of software and hardware, so worth investigating if there could be issues with unaligned data, which is more likely when the buffer is on the stack.

            Assignee:
            will.korteland@mongodb.com Will Korteland
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: