-
Type:
Improvement
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
None
-
None
-
None
-
None
-
None
Currently, passing an unsupported type in a schema to PyMongoArrow results in an obscure failure:
from pymongoarrow.api import Schema, find_pandas_all
from datetime import datetime
from pymongo import MongoClient
client = MongoClient()
client.db.data.delete_many({})
client.db.data.insert_many([
{'_id': 1, 'amount': 21, 'mac':'eee', 'last_updated': datetime(2020, 12, 10, 1, 3, 1)},
{'_id': 2, 'amount': 16, 'mac':'ddd', 'last_updated': datetime(2020, 7, 23, 6, 7, 11)},
{'_id': 3, 'amount': 3, 'mac':'eeeeee', 'last_updated': datetime(2021, 3, 10, 18, 43, 9)},
{'_id': 4, 'amount': 0, 'mac':'aaa', 'last_updated': datetime(2021, 2, 25, 3, 50, 31)}])
schema = Schema({'_id': pyarrow.int32(), 'amount': pyarrow.float64(), 'mac': pyarrow.string(), 'last_updated': datetime})
df = find_pandas_all(client.db.data, {'amount': {'$gt': 5}}, schema=schema)
Error:
df = find_pandas_all(client.db.data, {'amount': {'$gt': 5}}, schema=schema)
File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/api.py", line 142, in find_pandas_all
find_arrow_all(collection, query, schema=schema, **kwargs))
File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/api.py", line 59, in find_arrow_all
schema, codec_options=collection.codec_options)
File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/context.py", line 53, in from_schema
str_type_map = _get_internal_typemap(schema.typemap)
File "/home/fcdlab/.local/lib/python3.7/site-packages/pymongoarrow/types.py", line 70, in _get_internal_typemap
assert len(internal_typemap) == len(typemap)
AssertionError