|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 invalidate cached dns on fail
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/2ff810cd69074030293ae9fe78c9050736cdcb96
|
|
Author:
{'email': 'jesse@mongodb.com', 'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis'}
Message: CDRIVER-1972 don't log warning if connect fails
Now that we implement Happy Eyeballs, a successful IPv4-only connection
often involves a failed IPv6 attempt. Don't log a warning to stderr
about it.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/1ac262b6cb0dc6797b3655aee24192cf1b4c8ead
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 fix retired node failure
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/efc2042a6750fc092f2115f393ba934416ac89f8
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 update NEWS and docs
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/326ad94aa29ea677fdd35d52f820de8a3dedb3bc
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 test windows poll behavior
If windows connects to a host:port that isn't listened to, it takes
about one second for poll to receive a hangup.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/5fc74af941a96375047b3894db1358d8dca8b0af
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 implement Happy Eyeballs RFC
Summary: If a host has both ipv4 and ipv6 records, try ipv6 first.
If a connection can't be established after 250ms then try ipv4 in
parallel. If ipv6 fails immediately, try ipv4 immediately.
Whichever succeeds connection first cancels the other.
The successful DNS result is cached so subsequent connections
use the same DNS result without any possibility of delay. The
cache is invalidate after 10 minutes.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/5db4b701c55072eb190635ac3f773608df277784
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 initiate streams in async loop
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/af20e766610e5c2ed7250da071f967fa5df14231
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 fix DNS test on Travis
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/2a935728264f35849faf20f8fde688ec81f46392
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 fix socket handling
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/e87ad74d4cf874f330c6b8088a92013651dc2f84
|
|
There is some pending work after initial full support for IPv6 was merged.
- initial support does a simple parallel fan-out of IPv4 and IPv6 sockets attempting to connect. Whichever succeeds in responding to the ismaster command first wins. Instead we should change this to delay the IPv4 connection per the Happy Eyeballs RFC. And instead of waiting for an ismaster to respond closing other streams, we should close as soon as a connection succeeds (before any data is sent, including the TLS handshake)
- consider optimizing the search for other async commands to only search on the node in question. Async commands of a given node are all adjacent in the linked list.
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 initial full IPv6 support
- have a single server get scanned with multiple streams if DNS
results in multiple records. Use the first successful stream.
- refactor topology_scanner_node_t to remove acmd and stream
- only set topology_scanner_node_t.stream when a successful stream
is found
- allow mock server to bind to different sockaddr's
- add evergreen tasks for IPv4 and IPv6 addresses
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/756272f1fa33087a4c778e68003c0549f63c417d
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 do not shadow link symbol
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/fa03992efc6a62200c7ad4ef1968d54ee6dbbbdf
|
|
Author:
{'email': 'kevin.albertson@10gen.com', 'name': 'Kevin Albertson', 'username': 'kevinAlbs'}
Message: CDRIVER-1972 consolidate host parsing
Removes duplicated host parsing between mongoc_uri_t and
mongoc_host_list_t.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/265a95e652b8fe0a3596f74bc56851b45ac053c1
|
|
Kevin you can probably ignore most of the comments on this ticket. Here's the gist.
Currently, the driver chooses the address family for any hostname in its "seed list" (the hostnames in the URI) according to this logic:
/* this is in _mongoc_host_list_from_string */
|
/* like "[fe80::1]:27019" or ""[fe80::1]" */
|
if hostname starts with "[" and ends with "]" or has "]" and a port number:
|
address family = AF_INET6 /* IPv6 */
|
else:
|
address family = AF_INET /* IPv4 */
|
The driver later calls getaddrinfo with the address family it chose:
/* _mongoc_topology_do_blocking_scan() */
|
for each hostname:
|
call getaddrinfo with the chosen address family
|
begin a non-blocking connect to the first getaddrinfo result
|
|
start a timer to quit after connectTimeoutMS, default 10 seconds
|
|
/* this is mongoc_async_run() */
|
until we get a reply from all servers, or connectTimeoutMS expires:
|
run the event loop
|
What we should do instead is:
/* update _mongoc_host_list_from_string */
|
/* like "[fe80::1]:27019" or ""[fe80::1]" */
|
if hostname starts with "[" and ends with "]" or has "]" and a port number:
|
address family = AF_INET6 /* IPv6 */
|
else if hostname is obviously an IPv4 address like 1.2.3.4:
|
address family = AF_INET /* IPv4 */
|
else:
|
/* regular hostname like "mongodb.com" */
|
address family = AF_UNSPEC /* any protocol */
|
Then change how we scan the topology:
/* _mongoc_topology_do_blocking_scan() */
|
for each hostname:
|
call getaddrinfo with the chosen address family
|
begin non-blocking connections to ALL the getaddrinfo results!
|
|
start a timer to quit after connectTimeoutMS, default 10 seconds
|
|
/* this is mongoc_async_run() */
|
until we get a reply from all servers, or connectTimeoutMS expires:
|
run the event loop
|
if a connection succeeds to a server:
|
cancel other connections in progress to the same server with other addresses
|
cache the successful address family on the mongoc_topology_scanner_node_t
|
If the driver is operating in pooled mode, then it will need to make additional connections to each server - use the cached address family to speed up connections by using a known-good address family for the server.
Order of development will go something like:
- Update _mongoc_host_list_from_string to recognize IPv4 addresses, but still treat regular hostnames as AF_INET for now
- Treat regular hostnames as AF_UNSPEC, start connecting to all the addresses returned by getaddrinfo in parallel, ignore all but the first successful connection
- Teach the topology scanner to cancel all successful connections but the first, rather than ignoring them
- Use the address family that was cached on the mongoc_topology_scanner_node_t to speed up connections in pooled mode (mongoc_client_connect_tcp)
|
|
--- a/src/mongoc/mongoc-client.c 2016-08-10 12:32:09.000000000 -0700
|
+++ b/src/mongoc/mongoc-client.c 2017-03-24 12:40:59.038535139 -0700
|
@@ -308,6 +308,8 @@
|
|
|
switch (host->family) {
|
+
|
+ case AF_UNSPEC:
|
#if defined(AF_INET6)
|
case AF_INET6:
|
#endif
|
--- a/src/mongoc/mongoc-uri.c 2016-08-10 12:32:09.000000000 -0700
|
+++ b/src/mongoc/mongoc-uri.c 2017-03-24 12:41:29.132611988 -0700
|
@@ -96,7 +96,7 @@
|
} else {
|
bson_snprintf (link_->host_and_port, sizeof link_->host_and_port,
|
"%s:%hu", host, port);
|
- link_->family = strstr (host, ".sock") ? AF_UNIX : AF_INET;
|
+ link_->family = strstr (host, ".sock") ? AF_UNIX : AF_UNSPEC;
|
}
|
link_->host_and_port[sizeof link_->host_and_port - 1] = '\0';
|
link_->port = port;
|
--- a/src/mongoc/mongoc-topology-scanner.c 2016-03-16 15:48:40.000000000 -0700
|
+++ b/src/mongoc/mongoc-topology-scanner.c 2017-04-01 21:54:38.751702975 -0700
|
@@ -320,6 +320,11 @@
|
char portstr [8];
|
mongoc_host_list_t *host;
|
int s;
|
+ int64_t expire_at = 60 * MONGOC_DEFAULT_CONNECTTIMEOUTMS; /* 600 sec */
|
+ size_t num_streams_to_poll = 0;
|
+ mongoc_socket_poll_t *sds;
|
+ size_t nstreams = 2;
|
+ size_t i = 0;
|
|
ENTRY;
|
|
@@ -341,7 +346,7 @@
|
bson_set_error (error,
|
MONGOC_ERROR_STREAM,
|
MONGOC_ERROR_STREAM_NAME_RESOLUTION,
|
- "Failed to resolve '%s'",
|
+ "mongoc_topology_scanner_node_connect_tcp: Failed to resolve '%s'",
|
host->host);
|
RETURN (NULL);
|
}
|
@@ -351,6 +356,8 @@
|
mongoc_counter_dns_success_inc ();
|
}
|
|
+ sds = (mongoc_socket_poll_t *)bson_malloc(sizeof(*sds) * nstreams);
|
+
|
for (; node->current_dns_result;
|
node->current_dns_result = node->current_dns_result->ai_next) {
|
rp = node->current_dns_result;
|
@@ -364,23 +371,71 @@
|
continue;
|
}
|
|
- mongoc_socket_connect (sock, rp->ai_addr, (socklen_t)rp->ai_addrlen, 0);
|
+ if(nstreams == num_streams_to_poll) {
|
+ nstreams += 5;
|
+ sds = (mongoc_socket_poll_t *)bson_realloc (sds, sizeof (*sds) * nstreams);
|
+ }
|
|
- break;
|
+ if(0 == mongoc_socket_connect (sock, rp->ai_addr, (socklen_t)rp->ai_addrlen, 0))
|
+ {
|
+ /* lucky that already connected to an interface */
|
+ /* clean up rest of the stream connections for this node */
|
+ for(i = 0; i < num_streams_to_poll; ++i)
|
+ mongoc_socket_destroy (sds[i].socket);
|
+ bson_free(sds);
|
+ return mongoc_stream_socket_new (sock);
|
+ }
|
+
|
+ sds[num_streams_to_poll].socket = sock;
|
+ sds[num_streams_to_poll].events = POLLOUT;
|
+ num_streams_to_poll++;
|
+ sock = NULL;
|
+ }
|
+
|
+ /* If multiple dns records. Need to return only one stream */
|
+ sock = NULL;
|
+ if (num_streams_to_poll == 1)
|
+ sock = sds[0].socket; /* multi-socket poll helper will poll */
|
+
|
+ if (!sock && num_streams_to_poll) {
|
+ ssize_t ret = -1;
|
+
|
+ ret = mongoc_socket_poll(sds, num_streams_to_poll, expire_at);
|
+
|
+ if (ret > 0) {
|
+ /* an event happened, check if POLLOUT */
|
+ for (i = 0; i < num_streams_to_poll; i++) {
|
+ if (!sock && 0 != (sds[i].revents & POLLOUT)) /* only one stream need to be returned */
|
+ sock = sds[i].socket; /* connected stream */
|
+ else
|
+ /* clean up rest of the stream connections for this node */
|
+ mongoc_socket_destroy (sds[i].socket);
|
+ }
|
+ bson_free(sds);
|
+ num_streams_to_poll = 0;
|
+ sds = NULL;
|
+ }
|
}
|
|
if (!sock) {
|
bson_set_error (error,
|
MONGOC_ERROR_STREAM,
|
MONGOC_ERROR_STREAM_CONNECT,
|
- "Failed to connect to target host: '%s'",
|
+ "mongoc_topology_scanner_node_connect_tcp: Failed to connect to target host: '%s'",
|
host->host_and_port);
|
freeaddrinfo (node->dns_results);
|
node->dns_results = NULL;
|
node->current_dns_result = NULL;
|
+ for (i = 0; i < num_streams_to_poll; i++)
|
+ /* clean up all of the stream connections for this node */
|
+ mongoc_socket_destroy (sds[i].socket);
|
+ if(sds)
|
+ bson_free(sds);
|
RETURN (NULL);
|
}
|
|
+ if(sds)
|
+ bson_free(sds);
|
return mongoc_stream_socket_new (sock);
|
}
|
|
|
|
This is what I am using until we get official fix
if( results from the getaddrinfo list > 1) {
|
/* concurrently connect to all results for a given server */
|
non-blocking sockets and try all results concurrently for a given server and pick the protocol that succeeded quickest. ( new code)
|
}
|
else {
|
non-blocking sockets and try all results concurrently from all servers concurrently ( existing code)
|
}
|
|
|
Until we get official fix, I would use timeout only if > 1 addresses are returned. If only 1 address ( which should be the case most of the time) then fallback on exiting code logic.
|
|
The timeout is 0 so the topology scanner can use non-blocking sockets and try all servers concurrently. That's a huge optimization in the driver, which we want to keep, but it makes fixing this more complex than it appears. For C Driver 1.8 I intend to implement something close to "Happy Eyeballs", trying IPv4 and IPv6 concurrently and choosing the protocol that succeeded quickest.
|
|
Without fix mongoc_socket_connect() poll times out. But later there is another poll() that has a timeout of 9994.
1490982886.389783 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 126 <0.000033>
|
1490982886.389859 fcntl(126<socket:[955313]>, F_GETFL) = 0x2 (flags O_RDWR) <0.000024>
|
1490982886.389921 fcntl(126<socket:[955313]>, F_SETFL, O_RDWR|O_NONBLOCK) = 0 <0.000024>
|
1490982886.389982 setsockopt(126, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.000023>
|
1490982886.390039 connect(126, {sa_family=AF_INET, sin_port=htons(8191), sin_addr=inet_addr("X.X.X.X")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000044>
|
1490982886.390125 clock_gettime(CLOCK_MONOTONIC, {86521, 67552793}) = 0 <0.000024>
|
1490982886.390186 poll([{fd=126<socket:[955313]>, events=POLLOUT|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) <0.000025>
|
1490982886.390912 poll([{fd=25<socket:[955309]>, events=POLLOUT|POLLERR|POLLHUP}, {fd=125<socket:[955311]>, events=POLLOUT|POLLERR|POLLHUP}, {fd=126<socket:[955313]>, events=POLLOUT|POLLERR|POLLHUP}], 3, 9994) = 3 ([{fd=25, revents=POLLOUT}, {fd=125, revents=POLLOUT}, {fd=126, revents=POLLOUT}]) <0.000027>
|
So I think '10' sec timeout should be ok for mongoc-topology-scanner instead of '0' ?
|
|
Wondering if wait timeout '0' is too aggressive for mongoc_socket_connect ()?
1490974580.283405 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 76 <0.000033>
|
1490974580.283477 fcntl(76<socket:[901504]>, F_GETFL) = 0x2 (flags O_RDWR) <0.000029>
|
1490974580.283554 fcntl(76<socket:[901504]>, F_SETFL, O_RDWR|O_NONBLOCK) = 0 <0.000030>
|
1490974580.283627 setsockopt(76, SOL_TCP, TCP_NODELAY, [1], 4) = 0 <0.000029>
|
1490974580.283698 connect(76, {sa_family=AF_INET, sin_port=htons(8191), sin_addr=inet_addr("X.X.X.X")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000043>
|
1490974580.283792 clock_gettime(CLOCK_MONOTONIC, {78214, 961223578}) = 0 <0.000028>
|
1490974580.283862 poll([{fd=76<socket:[901504]>, events=POLLOUT|POLLERR|POLLHUP}], 1, 0) = 0 (Timeout) <0.000030>
|
1490974580.284121 shutdown(76, SHUT_RDWR) = 0 <0.000037>
|
1490974580.284196 close(76<socket:[901504]>) = 0 <0.000029>
|
|
|
I think the original fix for CDRIVER-1972 is correct just that mongoc-topology-scanner.c also need to attempt to connect all addresses just like mongoc-client.c is doing.
Original fix for CDRIVER-1972
--- src/mongoc/mongoc-client.c.orig 2016-03-30 13:25:38.000000000 -0700
|
+++ src/mongoc/mongoc-client.c 2017-03-25 10:41:19.215820495 -0700
|
@@ -303,6 +303,8 @@
|
|
|
switch (host->family) {
|
+
|
+ case AF_UNSPEC:
|
#if defined(AF_INET6)
|
case AF_INET6:
|
#endif
|
--- src/mongoc/mongoc-uri.c.orig 2016-03-30 13:25:39.000000000 -0700
|
+++ src/mongoc/mongoc-uri.c 2017-03-25 10:41:19.229820530 -0700
|
@@ -94,7 +94,7 @@
|
} else {
|
bson_snprintf (link_->host_and_port, sizeof link_->host_and_port,
|
"%s:%hu", host, port);
|
- link_->family = strstr (host, ".sock") ? AF_UNIX : AF_INET;
|
+ link_->family = strstr (host, ".sock") ? AF_UNIX : AF_UNSPEC;
|
}
|
link_->host_and_port[sizeof link_->host_and_port - 1] = '\0';
|
link_->port = port;
|
Additional fix for mongoc-topology-scanner.c to loop over all addresses.
--- src/mongoc/mongoc-topology-scanner.c.org 2017-03-25 10:43:53.972206542 -0700
|
+++ src/mongoc/mongoc-topology-scanner.c 2017-03-25 10:46:05.235533559 -0700
|
@@ -364,7 +364,26 @@
|
continue;
|
}
|
|
- mongoc_socket_connect (sock, rp->ai_addr, (socklen_t)rp->ai_addrlen, 0);
|
+ if (0 != mongoc_socket_connect (sock,
|
+ rp->ai_addr,
|
+ (socklen_t)rp->ai_addrlen,
|
+ 0)) {
|
+ char *errmsg;
|
+ char errmsg_buf[BSON_ERROR_BUFFER_SIZE];
|
+ char ip[255];
|
+
|
+ mongoc_socket_inet_ntop (rp, ip, sizeof ip);
|
+ errmsg = bson_strerror_r (
|
+ mongoc_socket_errno (sock), errmsg_buf, sizeof errmsg_buf);
|
+ MONGOC_WARNING ("Failed to connect to: %s:%d, error: %d, %s\n",
|
+ ip,
|
+ host->port,
|
+ mongoc_socket_errno(sock),
|
+ errmsg);
|
+ mongoc_socket_destroy (sock);
|
+ sock = NULL;
|
+ continue;
|
+ }
|
|
break;
|
}
|
|
|
No, it's not a configuration issue. All drivers can connect to "mongodb://localhost" on a Mac in the default configuration, even though Macs have IPv6 enabled on the loopback interface and MongoDB doesn't listen on IPv6 by default. So could the C Driver until CDRIVER-1988. The bug is as I described in CDRIVER-1988: we have to try all addresses returned by getaddrinfo, not just the first.
|
|
Related to the issue CDRIVER-1988
If I understanding correctly. Isn't this actually just a mongodb configuration issue if server is running without --ipv6 and you have IPv6 addresses in the network? Or the main problem is in backward incompatibility?
|
|
Author:
{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}
Message: Revert "CDRIVER-1972 Support for ipv6 hostnames"
8729c1448782481f392e4b51e513c14bb9736a5b
Fixes CDRIVER-1988.
Branch: r1.5
https://github.com/mongodb/mongo-c-driver/commit/0bce8b476ce0def8e9021b96436e816d2bf933dc
|
|
Author:
{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}
Message: Revert "CDRIVER-1972 Support for ipv6 hostnames"
8729c1448782481f392e4b51e513c14bb9736a5b
Fixes CDRIVER-1988.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/107dbb031d3d77662c2efb3a68997507845df0d1
|
|
Plus fix the line in test_mongoc_host_list_from_string:
ASSERT (host_list.family == AF_INET);
|
|
|
We need a different approach, see my comment at the end of CDRIVER-1988.
|
|
Author:
{u'name': u'Alexey Ponomarev', u'email': u'nine@yandex-team.ru'}
Message: CDRIVER-1972 Support for ipv6 hostnames
For cases when connection endpoint specified as hostname with ipv6
address only.
Branch: r1.5
https://github.com/mongodb/mongo-c-driver/commit/333cbc2cd2f54f3650f51c39a2490c28c355cc0f
|
|
Author:
{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}
Message: CDRIVER-1972 We explicitly know when its UDS but AF_UNSPEC might be ipv4 or ipv6
Branch: r1.5
https://github.com/mongodb/mongo-c-driver/commit/d25c2c64bbf73efa08c3701de3ea5b8f7b62b05d
|
|
Author:
{u'username': u'bjori', u'name': u'Hannes Magnusson', u'email': u'bjori@php.net'}
Message: CDRIVER-1972 We explicitly know when its UDS but AF_UNSPEC might be ipv4 or ipv6
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/9f1e63c625f9893fe185f1c98baec4ca359cecb2
|
|
Author:
{u'name': u'Alexey Ponomarev', u'email': u'nine@yandex-team.ru'}
Message: CDRIVER-1972 Support for ipv6 hostnames
For cases when connection endpoint specified as hostname with ipv6
address only.
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/8729c1448782481f392e4b51e513c14bb9736a5b
|
Generated at Wed Feb 07 21:13:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.