[CDRIVER-4725] Fetching data decimal data from CLOB looses precision Created: 15/Sep/23  Updated: 27/Oct/23  Resolved: 17/Oct/23

Status: Closed
Project: C Driver
Component/s: BSON
Affects Version/s: 1.21.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: arun mr Assignee: Kevin Albertson
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CDRIVER-4724 Fetching data decimal data from CLOB... Closed

 Description   

Summary

Fetching data decimal data  from CLOB looses precision

Environment

son lib 1.21.1

linuxamd64.

gcc 8

 

How to Reproduce

 

For  a data '{"a": 0.1234567890123456789012345, "b": "0.1234567890123456789012345" } after calling bson_init_from_json

 

when trying to access 'a ' we are getting 0.12345678901234568. we need full data. 

on analysis what we noticed is that the type of the data is BSON_TYPE_DOUBLE so 16 precisions are considered .

 

is this behaviour expected? is there any way to preserve the full precision. I know decfloat will, do but how can we control the conversion in  bson_init_from_json.

 

 

Additional Background

Please provide any additional background information that may be helpful in diagnosing the bug.



 Comments   
Comment by Kevin Albertson [ 17/Oct/23 ]

Looks like where the conversion from char to bson is causing the issue.

Parsing the JSON non-integer literals is expected to parse as double

If the number is a non-integer, parsers SHOULD interpret it as BSON Double.

The precision is limited to what a double can store.

If retaining the precision is needed, consider updating the JSON to store the value as a string or Decimal128 to retain the precision.

Issue is being closed since this does not appear to be a bug in the driver.

Comment by arun mr [ 17/Oct/23 ]

Thanks Kevin.

The type we are getting for this data "a" is 'BSON_TYPE_DOUBLE' . So we can't use bson_iter_decimal128.
In our code we are converting the data based on the type. We are converting different types like BSON_TYPE_DOUBLE, BSON_TYPE_INT64, BSON_TYPE_BOOL, BSON_TYPE_INT32. Looks like where the conversion from char to bson is causing the issue.

Comment by Kevin Albertson [ 13/Oct/23 ]

is there any " canonical_extended" version of bson_iter_init ??

"Canonical Extended" refers to the JSON representation.

In BSON, decimal128 is a separate type from double.

Extending the example above, a decimal128 can be iterated as follows:

   bson_iter_t iter;
   bson_iter_init_find (&iter, &b, "a");
   // Read value as Decimal128.
   bson_decimal128_t dec128;
   bson_iter_decimal128 (&iter, &dec128);

Comment by arun mr [ 13/Oct/23 ]

 IN my scenario the output from bson_init_from_json is used to create bson iterator and the iterator is used to fetch the data, like below.

const char from_json =

  "{\"a\" : {\"$numberDecimal\" : \"0.1234567890123456789012345\"}}";

bson_t b;

bson_error_t error;

bool ok = bson_init_from_json (&b, from_json, strlen (from_json), &error);

if (!ok)

{   printf ("error parsing: %s\n", error.message); }

bson_iter_t bsonIter;

bson_iter_init(&bsonIter, &b)
// doing data fetching and all.
bson_destroy (&b); .

is there any " canonical_extended" version of bson_iter_init ??

Comment by Kevin Albertson [ 12/Oct/23 ]

I expect the precision of 0.1234567890123456789012345 cannot be represented exactly in a double.

libbson parses JSON double values with strtod and prints with the format %.20g.

Use of PyMongo shows similar precision:

from bson import json_util
print(json_util.loads('{"a": 0.1234567890123456789012345 }'))
# Prints {'a': 0.12345678901234568}

 

know decfloat will, do but how can we control the conversion in bson_init_from_json.

The Decimal128 type can be represented in Extended JSON. Example with libbson:

const char *from_json =
    "{\"a\" : {\"$numberDecimal\" : \"0.1234567890123456789012345\"}}";
bson_t b;
bson_error_t error;
bool ok = bson_init_from_json (&b, from_json, strlen (from_json), &error);
if (!ok) {
    printf ("error parsing: %s\n", error.message);
}
char *to_json = bson_as_canonical_extended_json (&b, NULL /* length */);
printf ("got back: %s\n", to_json);
// Prints:
// got back: { "a" : { "$numberDecimal" : "0.1234567890123456789012345" } }
bson_free (to_json);
bson_destroy (&b);

Comment by PM Bot [ 15/Sep/23 ]

Hi arunmrmv@gmail.com, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:21:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.