[SERVER-14555] double floating point values are not retrieved exactly Created: 15/Jul/14  Updated: 10/Dec/14  Resolved: 16/Jul/14

Status: Closed
Project: Core Server
Component/s: Internal Client
Affects Version/s: 2.6.1
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Stefan Achatz Assignee: Thomas Rueckstiess
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

{
    mongo::DBClientConnection connection1, connection2, connection3;
    connection1.connect("localhost");
    connection2.connect("localhost");
    connection3.connect("localhost");
 
    for(double v1 = 0;; v1=nextafter(v1,1e32))
    {
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        mongo::BSONObj obj = BSON("_id" << 1 << "value" << v1);
        connection1.update("test.coll", qu, obj, /*upsert*/true, /*multi*/false);
      }
 
      double v2, v3;
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        std::auto_ptr<mongo::DBClientCursor> cursor = connection2.query("test.coll", qu);
        mongo::BSONObj obj = cursor->next();
        v2 = obj["value"].Double();
      }
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        std::auto_ptr<mongo::DBClientCursor> cursor = connection3.query("test.coll", qu);
        mongo::BSONObj obj = cursor->next();
        v3 = obj["value"].Double();
      }
 
      if(v1!=v2)
      {
        std::cout << "v1 = " << v1 << std::endl;
        std::cout << "v2 = " << v2 << std::endl;
      }
      if(v2!=v3)
      {
        std::cout << "v2 = " << v2 << std::endl;
        std::cout << "v3 = " << v3 << std::endl;
      }
    }
    return;
  }

output (exemplary, not deterministic):

v1 = 4.94066e-323
v2 = 4.44659e-323
v1 = 5.43472e-323
v2 = 4.44659e-323
v1 = 5.92879e-323
v2 = 4.44659e-323
v1 = 6.17582e-322
v2 = 6.12641e-322
v2 = 6.12641e-322
v3 = 6.17582e-322
v1 = 6.27463e-322
v2 = 6.22523e-322
v2 = 6.22523e-322
v3 = 6.27463e-322
...

Participants:

 Description   

1. Retrieving a double value from the database may differ from the originally stored value.
2. Two different connections may retrieve two different values for the same field.

This is a critical issue for us, because we have to trust that stored and retrieved numbers are exactly equal.

Versions: we use mongo db 2.6.1 server with 2.4.8 C++ driver (unfortunately we have severe problems to get the 2.6 C++ driver incorporated into our project, but this is different story...)



 Comments   
Comment by Thomas Rueckstiess [ 16/Jul/14 ]

Hi Stefan,

We can't see any evidence of a bug here and the SERVER project is only for reporting bugs and feature requests. Unfortunately we can't do code reviews or support answers in this project.

For this kind of question I think the mongodb-user (http://groups.google.com/group/mongodb-user) group would be better suited to get you some helpful answers, so I recommend you post your question there.

Regards,
Thomas

Comment by Stefan Achatz [ 16/Jul/14 ]

Hi Tyler,
thank you very much for taking care of my problem.

I confirm that calling lastError fixes the problem in the given test program.

In our project we call getLastErrorDetailed after every mongdb command and we evaluate and log the error (see details below). No error was ever reported.
Despite of this we have the reported inconsistency problem on double values.

I will now investigate in more detail our problem and try to come up with an updated test program which reproduces the error.
May I ask you to have a look on our error detection method and give a statement if it catches all possible errors?

bool MetaDBClient::update(mongo::Query qu, mongo::BSONObj obj)
{
  blog::LogStream warnLog("metadb.MetaDBClient.update", blog::warning);
  _lastException = "";
  try
  {
    (*_connection)->update(_namespace, qu, obj, /*upsert*/false, /*multi*/false);//lint !e730
    _lastError = (*_connection)->getLastErrorDetailed(/*fsync*/false, /*journal*/true, /*w propagation 0=primary, 1=one secondary, -1=majority*/  1, /*wtimeout, 0 is infinity*/0);
    mongo::BSONElement errField = _lastError["err"];
    if(!errField.eoo() && !errField.isNull())
    {
      warnLog << "Error during update:" << blog::end_of_line;
      warnLog << _lastError.jsonString() << blog::end_of_line;
      warnLog << "Query was:" << blog::end_of_line;
      warnLog << qu.toString() << blog::over;
      warnLog << "Update object was:" << blog::end_of_line;
      warnLog << obj.jsonString() << blog::over;
      return false;
    }
    return true;
  }
  catch(const mongo::DBException& e)
  {
    _lastException = e.toString();
    warnLog << "mongo error: " << _lastException << blog::over;
  }
  return false;
}
 

Comment by Tyler Brock [ 15/Jul/14 ]

I have confirmed the legacy driver simply works out of the box with your program (as expected) and that the 26compat version of the driver works with my suggested change:

#include <iostream>
#include "include/mongo/client/dbclient.h"
 
int main() {
    mongo::DBClientConnection connection1, connection2, connection3;
    connection1.connect("localhost");
    connection2.connect("localhost");
    connection3.connect("localhost");
 
    for(double v1 = 0;; v1=nextafter(v1,1e32))
    {
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        mongo::BSONObj obj = BSON("_id" << 1 << "value" << v1);
        connection1.update("test.coll", qu, obj, /*upsert*/true, /*multi*/false);
        connection1.getLastError();
      }
 
      double v2, v3;
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        std::auto_ptr<mongo::DBClientCursor> cursor = connection2.query("test.coll", qu);
        mongo::BSONObj obj = cursor->next();
        v2 = obj["value"].Double();
      }
      {
        mongo::BSONObj qu = BSON("_id" << 1);
        std::auto_ptr<mongo::DBClientCursor> cursor = connection3.query("test.coll", qu);
        mongo::BSONObj obj = cursor->next();
        v3 = obj["value"].Double();
      }
 
      if(v1!=v2)
      {
        std::cout << "v1 = " << v1 << std::endl;
        std::cout << "v2 = " << v2 << std::endl;
      }
      if(v2!=v3)
      {
        std::cout << "v2 = " << v2 << std::endl;
        std::cout << "v3 = " << v3 << std::endl;
      }
    }
    return 0;
}

Comment by Tyler Brock [ 15/Jul/14 ]

Hey Stefan,

You just need to call {{ connection1.getLastError(); }} right after you do the update.

With the 26compat version of the driver writes are not acknowledged by default so you need to call getLastError to ensure that the write is completed before querying. Without that, there are no guarantees the write happens before the first query, before the second query, or at all.

With the legacy version of the driver, we have updated the code such that acknowledged write concern is the default so the program works as expected by default.

Generated at Thu Feb 08 03:35:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.