Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-1985

Java binding doesn't encode the integer 8256 correctly

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT2.7.0
    • Labels:
      None
    • # Replies:
      16
    • Last comment by Customer:
      true

      Description

      Since 2.5.0, the C library makes a special case for encoding the integer value "8256" which is (2^13 + 2^16). The java library doesn't make the same special case, so it is possible to corrupt the database state by writing that value into it (see attached test case).

      Were there and special precautions that WiredTiger takes when upgrading versions that going from 2.4 -> 2.5 re-encoded any values that changed? How would similar encoding changes in the future be handled?

      Why doesn't the java binding use the C wiredtiger_struct_pack instead of reimplementing the same algorithm? At a very quick glance it looks like the python binding does the same thing, so it is likely similarly affected.

      1. IntegerPackingBug.java
        1.0 kB
        Greg Rogers

        Issue Links

          Activity

          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@ddanderson.com'}

          Message: WT-1985. Added teardown() call so multiple tests don't fail.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/dade5741299d481bc4b06e79d5a9de2e51b86a18

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@ddanderson.com'} Message: WT-1985 . Added teardown() call so multiple tests don't fail. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/dade5741299d481bc4b06e79d5a9de2e51b86a18
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@ddanderson.com'}

          Message: WT-1985. Fixed a comment about 'U' unpacking so there is no confusion.
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/97befce992e8fe26e6a8996ea9eecc957056d02b

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'ddanderson', u'name': u'Don Anderson', u'email': u'dda@ddanderson.com'} Message: WT-1985 . Fixed a comment about 'U' unpacking so there is no confusion. Branch: develop https://github.com/wiredtiger/wiredtiger/commit/97befce992e8fe26e6a8996ea9eecc957056d02b
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'}

          Message: Merge pull request #2055 from wiredtiger/integer-packing-java

          WT-1985. Integer packing and other fixes for Java
          Branch: develop
          https://github.com/wiredtiger/wiredtiger/commit/57a9f380a3200729966502176f4c1cb881354d8c

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'agorrod', u'name': u'Alex Gorrod', u'email': u'alexander.gorrod@mongodb.com'} Message: Merge pull request #2055 from wiredtiger/integer-packing-java WT-1985 . Integer packing and other fixes for Java Branch: develop https://github.com/wiredtiger/wiredtiger/commit/57a9f380a3200729966502176f4c1cb881354d8c
          Hide
          donald.anderson Donald Anderson added a comment -

          Greg Rogers, the most recent commit to the develop branch should fix this problem, as well as some other issues related to packing/unpacking. The Python interface has been updated as well.

          It's a good question why this code is language-specific. One reason is that the base (C) API is coded with variable arguments. Translating that interface to Java, Python and potentially other languages in the context of a wrapper generator like SWIG opens its own can of worms - we'd be trading one problem for another. A second reason may be performance; assembling and extracting individual fields is a lightweight operation, and crossing the language boundary can add additional cost. I don't think either of these reasons is entirely convincing - this may be a decision we can revisit in the future.

          Show
          donald.anderson Donald Anderson added a comment - Greg Rogers , the most recent commit to the develop branch should fix this problem, as well as some other issues related to packing/unpacking. The Python interface has been updated as well. It's a good question why this code is language-specific. One reason is that the base (C) API is coded with variable arguments. Translating that interface to Java, Python and potentially other languages in the context of a wrapper generator like SWIG opens its own can of worms - we'd be trading one problem for another. A second reason may be performance; assembling and extracting individual fields is a lightweight operation, and crossing the language boundary can add additional cost. I don't think either of these reasons is entirely convincing - this may be a decision we can revisit in the future.
          Hide
          donald.anderson Donald Anderson added a comment -

          This problem, and other encoding/decoding issues discovered in Java and Python, have been fixed in recent commits to WT/develop.

          Show
          donald.anderson Donald Anderson added a comment - This problem, and other encoding/decoding issues discovered in Java and Python, have been fixed in recent commits to WT/develop.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                1 year, 36 weeks, 1 day ago
                Date of 1st Reply: