-
Type: Bug
-
Resolution: Duplicate
-
Priority: Major - P3
-
None
-
Affects Version/s: 10.1.0, 10.1.1
-
Component/s: None
-
None
One of the recently added features (the spark.mongodb.write.convertJson option) handles the Bson specific data-types, like ObjectId, properly. But, it automatically converts other fields of other types as well, which is not the desired behaviour in every case.
For example, in our use-case the product code field almost always is an integer value, and we store it as string (the type handling is done in Spark), the aforementioned feature automatically converts this field to integer type and it doesn’t offer any other option to override/affect this behaviour.
This was not the case with the v3 of this connector, and it is blocking our organisation from using the newest version.
- duplicates
-
SPARK-388 Datatype overwritten while insertion into Mongo on using mongo-spark-connector 10.1.0v
- Closed