According to the crypto team, the maximum precision that can be specified for range-indexed encrypted decimal128 types should be 6176, since the smallest positive subnormal decimal128 value is 1 x 10^-6176. Currently, the libmongocrypt enforces a max of 6182.

Quote from slack conversation with zachary.espiritu@mongodb.com:

Just to follow up on this, I checked with Tarik and we expect the largest allowable precision for decimal128 in QE ranges to be 6176 (because the smallest positive subnormal decimal128 is 1 x 10^{-6176}). It's not clear to me if this "6176" number is explicitly stated in the IEEE specification in the context of subnormal numbers but the way this is derived is from the fact that there are 34 decimal digits of significand means the smallest number you can represent with the significand alone is 1x10^{-33}, and the exponent range is [-6143, 6144] so the smallest positive subnormal is 1 x 10^{-33} x 10^{-6143} = 10^{-6176}.

(There is a similar precision <= 324 bounds check being done in both the 10gen/mongo and libmongocrypt getTypeInfoDouble conversions from double to its OST representation, which does match with what we expect it to be, namely the exponent in ~4.9x10^{-324}, the smallest positive subnormal double)

Update (06-24-2024): After discussing with the crypto team, we find that these upper bounds (for both decimal and double) don't really make sense with respect to the purpose of the "precision" parameter, hence it was decided that it would be better to remove these assertions. On a related note, if the precision, min, and max parameters, combined, don't result in a more performant encoding of the entire domain, then it was decided that it would be better to error, rather than silently fall back to the default encoding (see SERVER-91571).