Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-7765

Fix signed integer overflow in intpack-test3

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • WT10.0.1, 4.4.8, 5.0.2, 5.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • 2
    • Storage - Ra 2021-07-12

      The UBSAN builder found the following signed integer overflow error:

      ../../../test/packing/intpack-test3.c:105:13: runtime error: signed integer overflow: 9223372036854775807 + 1025 cannot be represented in type 'long'
      ../../../test/packing/intpack-test3.c:106:40: runtime error: signed integer overflow: 9223372036854775807 + 1025 cannot be represented in type 'long'

      Relevant code in intpack-test3.c:

      void
      test_spread(int64_t start, int64_t before, int64_t after)
      {
          int64_t i;    printf("Testing range: %" PRId64 " to %" PRId64 ". Spread: % " PRId64 "\n", start - before,
            start + after, before + after);
          for (i = start - before; i < start + after; i++)
              test_value(i);
      }

      The error occurs when the value INT64_MAX is passed as the start parameter to the test_spread function which causes the start+after value to overflow. Perhaps using in INT64_MAX-1025 could be a suitable fix.

            Assignee:
            jeremy.thorp@mongodb.com Jeremy Thorp
            Reporter:
            tammy.bailey@mongodb.com Tammy Bailey (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: