[CDRIVER-4593] C Driver fails to validate double type connectionId during handshake process Created: 23/Mar/23  Updated: 28/Oct/23  Resolved: 27/Mar/23

Status: Closed
Project: C Driver
Component/s: libmongoc
Affects Version/s: 1.23.1, 1.23.2
Fix Version/s: 1.23.3

Type: Bug Priority: Major - P3
Reporter: Kondaiah Valagonda Assignee: Unassigned
Resolution: Fixed Votes: 1
Labels: Bug, linux
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File connectionId_as_double.png     PNG File connectionId_as_int32.png    
Issue Links:
Problem/Incident
is caused by SERVER-75293 Different return types for the connec... Closed
Related
is related to CDRIVER-4502 libmongoc expects connectionId in hel... Closed
Quarter: FY24Q1

 Description   

Summary

Mongo with higher volume of connections may return connectionId as double during the handshake process. The C(1.23.2) & C++(3.7.x)drivers fail to do a type check for double type connectionId in hello/ismaster response and hence fails to connect to mongo.

This happens in clusters with a high volume of connections created at a higher rate and/or without any connection pooling mechanism.

db.isMaster() response from mongo shell:

    db.isMaster()
    {
        "ismaster" : true,
        "msg" : "isdbgrid",
      ...
      ...
      "connectionId" : 3177689371,
      ...
      ...
    }

 

db.isMaster() response from python driver:

 

{
   'ok': 1.0, 
   'msg': 'isdbgrid', 
   'ismaster': True, 
   ...
   ...  
   'connectionId': 3177722651.0, <==== double value
   ...
   ...
}

 

The C driver fails to check connectionId value, 3177722651.0 with BSON_ITER_HOLDS_INT

Environment

 C / CXX driver versions:

 

{"name":"mongoc / mongocxx","version":"1.23.1 / 3.7.0"}
{"name":"mongoc / mongocxx","version":"1.23.2 / 3.7.0"} 

 C / CXX compiler and version:

 

gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
c++11 & c++17

 

Mongo:

Sharded Cluster with v4.4.14

How to Reproduce

It's difficult to reproduce the issue. Need to create connections at a higher rate to overflow int32 value of connectionId.

Temporary Fix

Restarting the mongo service would reset the connectionId counter.

 

 



 Comments   
Comment by Kondaiah Valagonda [ 27/Mar/23 ]

Thank you for the update, Kevin

Comment by Kevin Albertson [ 27/Mar/23 ]

Thank you for the fix konda.valagonda@gmail.com. C driver 1.23.3 is planned for release next Tuesday, April 4.

Comment by Kondaiah Valagonda [ 27/Mar/23 ]

@kevin.albertson@mongodb.com Thanks for accepting the PR (https://github.com/mongodb/mongo-c-driver/pull/1222) Could you provide the ETA on the C-Driver v1.23.3?

Comment by Githook User [ 27/Mar/23 ]

Author:

{'name': 'Kondaiah Valagonda', 'email': '128669292+kvalagonda@users.noreply.github.com', 'username': 'kvalagonda'}

Message: CDRIVER-4593: Handle double type connectionId (#1222)

  • use scopes in test_server_description_connection_id
  • use cleanup instead of reset
  • use ASSERT_CMPINT64
  • add test for double

---------

Co-authored-by: Kevin Albertson <kevin.albertson@mongodb.com>
Branch: r1.23
https://github.com/mongodb/mongo-c-driver/commit/2b370c8825240495c8e5752199a343369a64fd0e

Comment by Githook User [ 27/Mar/23 ]

Author:

{'name': 'Kondaiah Valagonda', 'email': '128669292+kvalagonda@users.noreply.github.com', 'username': 'kvalagonda'}

Message: CDRIVER-4593: Handle double type connectionId (#1222)

  • use scopes in test_server_description_connection_id
  • use cleanup instead of reset
  • use ASSERT_CMPINT64
  • add test for double

---------

Co-authored-by: Kevin Albertson <kevin.albertson@mongodb.com>
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/e040cebc48e0e871a7ed9c1bc23e4abb65a373fd

Comment by Kondaiah Valagonda [ 27/Mar/23 ]

Hello,

We've submitted the below PR for the C-driver fix. We've tested the fix, and the C-driver is able to handle Int32, Int64, and Double types of connectionIds without any issues.

https://github.com/mongodb/mongo-c-driver/pull/1222

Also, The below SERVER ticket has server-side(mongo) bug details. 

https://jira.mongodb.org/browse/SERVER-75293

Either the [PR|https://github.com/mongodb/mongo-c-driver/pull/1222] or the SERVER ticket has to be fixed. We would prefer to have the fix available in the C-driver first. Please let us know.

 

Thanks, KV

Comment by Kondaiah Valagonda [ 24/Mar/23 ]

Thank you Jeremy Mikola. The attached Wireshark tcpdump dump also shows that connectionId coming as a Double from a mongos that had a bigger connectionId value, and Int32 from a restarted mongos service. We will submit a SERVER ticket and the PR also with the libmongoc fix.

 

Comment by Jeremy Mikola [ 23/Mar/23 ]

Thanks for following up with extended JSON output.

Since you're connected to a v4.4.14 sharded cluster, I think the relevant code path may be:

It's still not clear to me where the floating point value is coming from, but if you're confident that the server is responding with a double then I think this may warrant a SERVER ticket (feel free to link back to this for additional context).

AFAIK, drivers have only ever expected integer values for connectionId. I don't think changing libmongoc to accept a double would be the best course of action.

Comment by Kondaiah Valagonda [ 23/Mar/23 ]

@jeremy Mikola It's the real double connectionId coming from the mongo hello response and below is the C canonical extended json output of hello(). 

 

hello_response={ "ismaster" : true, "msg" : "isdbgrid", "helloOk" : true, "maxBsonObjectSize" : { "$numberInt" : "16777216" }, "maxMessageSizeBytes" : { "$numberInt" : "48000000" }, "maxWriteBatchSize" : { "$numberInt" : "100000" }, "localTime" : { "$date" : { "$numberLong" : "1679542351699" } }, "logicalSessionTimeoutMinutes" : { "$numberInt" : "30" }, 
 
"connectionId" : { "$numberDouble" : "3180723594.0" },
 
"maxWireVersion" : { "$numberInt" : "9" }, "minWireVersion" : { "$numberInt" : "0" }, "topologyVersion" : { "processId" : { "$oid" : "630432bd25644cef24419f952" }, "counter" : { "$numberLong" : "0" } }, "ok" : { "$numberDouble" : "1.0" }, "operationTime" : { "$timestamp" : { "t" : 1679542351, "i" : 1713 } }, "$clusterTime" : { "clusterTime" : { "$timestamp" : { "t" : 1679542351, "i" : 1713 } }, "signature" : { "hash" : { "$binary" : { "base64" : "89m/k2GPfisrfyBQYypFumhFwYw=", "subType" : "00" } }, "keyId" : { "$numberLong" : "7173351955694092622" } } } }

Used the below snippet in mongoc-server-description.c 

//snippet to print details using canonical extend json. 
char *str = bson_as_canonical_extended_json (&sd->last_hello_response, NULL); 
fprintf (stdout, "hello_response=%s\n", str); 

Yes, CDRIVER-4502  is already fixed in libmongoc 1.23.1 & 1.23.2, but this is totally a new issue with 1.23.1/2.

 

Comment by Jeremy Mikola [ 23/Mar/23 ]

I think it's unlikely that the server is returning a double here. Rather, you're seeing a double due to pymongo's JSON output. You might get more accurate results by having pymongo output canonical extended JSON (see: json_util), as that would report type information for 64-bit integers (see: Conversion Table in the Extended JSON spec).

The "connectionId" field in the hello/isMaster response is appended in cluster_hello_cmd.cpp. bsonobjbuilder.h does have several overloads for the appendNumber() method, but the argument here is coming from getConnectionId() and is likely a long long according to the typedef earlier in the file.

Having said that, this looks related to CDRIVER-4502, which was fixed in libmongoc 1.22.2 and 1.23.1.

Generated at Wed Feb 07 21:21:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.