[SERVER-27012] Cannot add hosts to replica set on Docker 1.13.0-rc1 swarm network. Created: 13/Nov/16 Updated: 13/Nov/16 Resolved: 13/Nov/16 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | 3.2.10, 3.4.0-rc2 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Daniel Shannon | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | replicaset | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Participants: |
| Description |
|
Hey Mongoids,
The three containers come up successfully, usually on hosts that are coincidentally physically separate. From each container, I'm able to ping the others (in my configuration, they typically resolve to the IP addresses 10.0.72.2, 10.0.72.4, and 10.0.72.6) and connect to them with the Mongo shell using their hostnames. In mongo1, I initiate a replica set and update the first member's hostname:
All seems to be well, but begins to go awry when I attempt to add another service to the set:
The only log line that seemed pertinent was this one:
For the life of me, I can't figure out what causes this check to fail---the network seems functional and the services are isolated and distinct in every conceivable way. I'm working on rolling back to Docker 1.12 instead of the 1.13 release candidate to see if some change in their networking stack caused this, but I'd be grateful to know if you had any insight into what might cause MongoDB to believe that hosts are duplicates of one another. Thanks so much, in advance, for your thoughts! |
| Comments |
| Comment by Daniel Shannon [ 13/Nov/16 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Issue created on Docker's GitHub here: https://github.com/docker/docker/issues/28358. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Ramon Fernandez Marina [ 13/Nov/16 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This data in the description above looks suspicious:
While in the output you provided the first host shows as mongo1:27017. My guess is that someting in the 1.13 version is making hosts mongo1 and mongo2 to resolve to the same host, and triggering the rs.add() check and error message. You may want to try running rs.initiate() in the mongo2 host and see if rs.conf() also shows mongo2 as "host" : "1e0b80e3f356:27017". I'm going to close this ticket, since this appears to be an issue with host resolution on the docker side and the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag, where your question will reach a larger audience. Regards, | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Daniel Shannon [ 13/Nov/16 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
After rolling back to 1.12, I was able to create a replica set as expected:
I'll certainly kick this to the Docker folks, but it would still be great if you all had any insight into exactly what MongoDB was seeing here that threw it. |