Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-12067

Improve/Fix CRC calculation and testing on zSeries

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • StorEng - Defined Pipeline

      While working on WT-11899 to add a wiredtiger_crc32c_with_seed_func API, I found some issues with zSeries. I am creating this ticket as the follow on work:

      The wiredtiger_crc32c_with_seed_func API that I added in WT-11899 is supposed to act like wiredtiger_crc32c_func, except that it takes a starting seed. The objective is to call the new API on consecutive chunks of a larger memory buffer, carrying the calculated CRC as the starting seed for the next chunk. The resulting CRC calculation should be the same if we calculate on the whole buffer in one go. These API actually return function pointers to an internal implementation, which is either software defined in checksum/software or hardware accelerated in checksum/<platform>.

      In the time I had, I could not get the hardware routines for the zSeries to behave correctly when calculating the checksum in chunks. This is because of the big endianness of the platform, where calculation is done over the buffer right to left, whereas when divided into chunks, the chunks go left to right. Hence, I forced wiredtiger_crc32c_with_seed_func to only use the software routine. We have a software CRC calculation routine common to all platforms in __wt_checksum_sw and __wt_checksum_with_seed_sw. But for zSeries we also have __wt_crc32c_le. I found that __wt_crc32c_le gave the correct result, but __wt_checksum_with_seed_sw doesn't. We need to investigate why.

      For now, I am using __wt_crc32c_le - so applications will get a good result. But the tests in test/csuite/wt2695_checksum compare the results of various routines among each other and hence fail. I have disabled the failing test for __wt_checksum_with_seed_sw for the zSeries for the time being.

      Hence, the main task with this ticket is to:

      1. Check if we can implement hardware acceleration for __wt_checksum_with_seed_hw for the zSeries.
      2. Investigate why __wt_crc32c_le gives the correct result for CRC over chunks while __wt_checksum_with_seed_sw doesn't. Fix as needed and re-enable tests disabled for zSeries in test/csuite/wt2695_checksum.
      3. Irrespective of this work related to CRC, I found that the existing CRC tests are not currently running on zSeries - I created WT-12057 to look into it, so this ticket needs to be aware of the current lack of testing for CRC calculation on zSeries.
      4. test/unittest/tests/test_crc32.cpp currently fails on the zSeries (even before the changes from WT-11899}} - so that needs to be fixed.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            sulabh.mahajan@mongodb.com Sulabh Mahajan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: