Uploaded image for project: 'Python Driver'
  1. Python Driver
  2. PYTHON-1430

regression in bson.json_util between python 2 and python 3

    • Type: Icon: Bug Bug
    • Resolution: Won't Fix
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.5.1
    • Component/s: BSON
    • Labels:
    • Environment:
      OSX, pymongo 3.5.1

      There appears to be a regression in the way bson handles `numpy.int64` and `numpy.float64` values.

      Works in 2.7

      {{In [2]: sys.version
      Out[2]: '2.7.11 |Anaconda 2.0.1 (x86_64)| (default, Dec 6 2015, 18:57:58) \n[GCC 4.2.1 (Apple Inc. build 5577)]'

      In [3]: import bson.json_util as bju

      In [4]: import numpy as np

      In [5]: bju.dumps(

      {'a': np.int64(12345)}

      )
      Out[5]: '

      {"a": 12345}

      '
      }}

      Fails in 3.6

      {{In [69]: import sys

      In [70]: sys.version
      Out[70]: '3.6.2 |Anaconda custom (x86_64)| (default, Jul 20 2017, 13:14:59) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]'

      In [71]: import bson.json_util as bju

      In [72]: import numpy as np

      In [73]: bju.dumps(

      {'a': np.int64(12345)}

      )
      ---------------------------------------------------------------------------
      TypeError Traceback (most recent call last)
      <ipython-input-73-03e5f501a15a> in <module>()
      ----> 1 bju.dumps(

      {'a': np.int64(12345)}

      )

      /Users/shankari/OSS/anaconda/envs/py36/lib/python3.6/site-packages/bson/json_util.py in dumps(obj, *args, **kwargs)
      401 """
      402 json_options = kwargs.pop("json_options", DEFAULT_JSON_OPTIONS)
      --> 403 return json.dumps(_json_convert(obj, json_options), *args, **kwargs)
      404
      405

      /Users/shankari/OSS/anaconda/envs/py36/lib/python3.6/json/_init_.py in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
      229 cls is None and indent is None and separators is None and
      230 default is None and not sort_keys and not kw):
      --> 231 return _default_encoder.encode(obj)
      232 if cls is None:
      233 cls = JSONEncoder

      /Users/shankari/OSS/anaconda/envs/py36/lib/python3.6/json/encoder.py in encode(self, o)
      197 # exceptions aren't as detailed. The list call should be roughly
      198 # equivalent to the PySequence_Fast that ''.join() would do.
      --> 199 chunks = self.iterencode(o, _one_shot=True)
      200 if not isinstance(chunks, (list, tuple)):
      201 chunks = list(chunks)

      /Users/shankari/OSS/anaconda/envs/py36/lib/python3.6/json/encoder.py in iterencode(self, o, _one_shot)
      255 self.key_separator, self.item_separator, self.sort_keys,
      256 self.skipkeys, _one_shot)
      --> 257 return _iterencode(o, 0)
      258
      259 def _make_iterencode(markers, _default, _encoder, _indent, _floatstr,

      /Users/shankari/OSS/anaconda/envs/py36/lib/python3.6/json/encoder.py in default(self, o)
      178 """
      179 raise TypeError("Object of type '%s' is not JSON serializable" %
      --> 180 o._class.name_)
      181
      182 def encode(self, o):

      TypeError: Object of type 'int64' is not JSON serializable

      }}

      This complicates interfacing with pandas because pandas returns values in `numpy.int64` and `numpy.float64`.

            Assignee:
            bernie@mongodb.com Bernie Hackett
            Reporter:
            shankari K. Shankari
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: