Major - P3 Precision and scale for an Array[Decimal128] field is inferred based on the value at the 0th index of the array. This results in 'null' values for all Decimal128 subfields having precision greater than the first element of the array.
Example
Source data - Mongo collection holding single record
- _id = "xxx" (String)
- arr = [1.4, 12.45] (Decimal128)
Schema inferred for above collection by MongoSpark read is
| – _id: string (nullable = true) | |
| – arr: array (nullable = true) | |
| – element: decimal(2,1) (containsNull = true) |
Here the second value (12.45) becomes null as it has higher precision.