Uploaded image for project: 'pymongoarrow'
  1. pymongoarrow
  2. ARROW-35

Improve error message when schema contains an unsupported type

    • Type: Icon: Improvement Improvement
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 0.3.0
    • Affects Version/s: None
    • Component/s: None
    • Labels:

      Currently, passing an unsupported type in a schema to PyMongoArrow results in an obscure failure:

        from pymongoarrow.api import Schema, find_pandas_all
        from datetime import datetime
        from pymongo import MongoClient
      
        client = MongoClient()
        client.db.data.delete_many({})
        client.db.data.insert_many([
            {'_id': 1, 'amount': 21, 'mac':'eee', 'last_updated': datetime(2020, 12, 10, 1, 3, 1)},
            {'_id': 2, 'amount': 16,  'mac':'ddd', 'last_updated': datetime(2020, 7, 23, 6, 7, 11)},
            {'_id': 3, 'amount': 3,  'mac':'eeeeee', 'last_updated': datetime(2021, 3, 10, 18, 43, 9)},
            {'_id': 4, 'amount': 0, 'mac':'aaa',  'last_updated': datetime(2021, 2, 25, 3, 50, 31)}])
      
        schema = Schema({'_id': pyarrow.int32(), 'amount': pyarrow.float64(), 'mac': pyarrow.string(), 'last_updated': datetime})
        df = find_pandas_all(client.db.data, {'amount': {'$gt': 5}}, schema=schema)
      

      Error:

          df = find_pandas_all(client.db.data, {'amount': {'$gt': 5}}, schema=schema)
        File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/api.py", line 142, in find_pandas_all
          find_arrow_all(collection, query, schema=schema, **kwargs))
        File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/api.py", line 59, in find_arrow_all
          schema, codec_options=collection.codec_options)
        File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/context.py", line 53, in from_schema
          str_type_map = _get_internal_typemap(schema.typemap)
        File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/types.py", line 70, in _get_internal_typemap
          assert len(internal_typemap) == len(typemap)
      AssertionError
      

            Assignee:
            steve.silvester@mongodb.com Steve Silvester
            Reporter:
            prashant.mital Prashant Mital (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: