Just upgraded to 1.11 and suddenly everything is failing with
491943 File "/opt/beyond/code/api/beyond/../../externals/pymongo/connection.py", line 324, in _init_
491945 File "/opt/beyond/code/api/beyond/../../externals/pymongo/connection.py", line 542, in __find_node
491946 for candidate in self.__nodes:
491947 RuntimeError: Set changed size during iteration
The issue is in connections.py when connecting to a replica set. __find_node() iterates over the node set and calls __try_node() on each node:
522 def __find_node(self):
542 for candidate in self.__nodes:
543 node = self.__try_node(candidate)
544 if node:
545 return node
486 def __try_node(self, node):
505 # Replica Set?
506 if len(self.__nodes) > 1:
507 if "hosts" in response:
509 for h in response["hosts"]])
If the host names specified when creating the connection are not FQDNs, or if the name of every replica set member is not specified, then __try_node() modifies the set of nodes which are being iterated over which is illegal. Python sets are not safe for modification during iteration.
My issue was cause because the host names specified are not FQDNs so they do not match exactly the names returned from mongodb.ismaster() resulting in the set being modified which yields the error. FYI we were not using FQDNs because everything using mongo is internal on a secure net inside our data center.