Minor - P4 Adding new entries to bson._ENCODERS is not thread safe because we also iterate it.
The attached repro script, bson-not-thread-safe.py
, shows the issue. When multiple threads are encoding custom classes one will fail with a RuntimeError:
Exception while encoding <class '__main__.MyInt_0_4'>: dictionary changed size during iteration Exception in thread Thread-1: Traceback (most recent call last): File "/usr/local/Cellar/pypy/5.10.0_1/libexec/lib-python/2.7/threading.py", line 797, in __bootstrap_inner self.run() File "/usr/local/Cellar/pypy/5.10.0_1/libexec/lib-python/2.7/threading.py", line 750, in run self.__target(*self.__args, **self.__kwargs) File "bson-not-thread-safe.py", line 9, in target bson.BSON.encode({'my_int': my_int()}) File "/Users/shane/git/mongo-python-driver/bson/__init__.py", line 1049, in encode return cls(_dict_to_bson(document, check_keys, codec_options)) File "/Users/shane/git/mongo-python-driver/bson/__init__.py", line 831, in _dict_to_bson check_keys, opts)) File "/Users/shane/git/mongo-python-driver/bson/__init__.py", line 816, in _element_to_bson return _name_value_to_bson(name, value, check_keys, opts) File "/Users/shane/git/mongo-python-driver/bson/__init__.py", line 786, in _name_value_to_bson for base in _ENCODERS: RuntimeError: dictionary changed size during iteration
We could fix this by:
1) remove the logic to cache custom subtypes, or
2) Instead of modifying _ENCODERS we can add new entries to another dict, like _CACHED_ENCODERS. This would require a second dict lookup for each encoded value, or
3) Instead of iterating over the keys of _ENCODERS (which may change), iterate over a global copy of the initial keys.
3) seems like the best solution to me.
Note: This issue is only present went the C extensions are not installed.