m33058| Mon Dec 21 13:21:52 connection accepted from 127.0.0.1:59544 #1
shell: started mongo program /home/buildslave/buildslave/Linux_32bit/mongo/mongod --nohttpinterface --noprealloc --bind_ip 127.0.0.1 --smallfiles --port 54675 --dbpath /data/db/repl6tests-1-slave --slave --source 127.0.0.1:33058 --oplogSize 1
m54675| Mon Dec 21 13:21:52 Mongo DB : starting : pid = 2105 port = 54675 dbpath = /data/db/repl6tests-1-slave master = 0 slave = 1 32-bit
m54675|
m54675| ** NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data
m54675| ** see http://blog.mongodb.org/post/137788967/32-bit-limitations for more
m54675|
m54675| Mon Dec 21 13:21:52 db version v1.3.0-, pdfile version 4.5
m54675| Mon Dec 21 13:21:52 git version: 499cb647b65ca9d1b0cbdc27f72359b30bf2bf39
m54675| Mon Dec 21 13:21:52 sys info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_37
m54675| Mon Dec 21 13:21:52 waiting for connections on port 54675
m54675| Mon Dec 21 13:21:53 connection accepted from 127.0.0.1:33904 #1
shell: started mongo program /home/buildslave/buildslave/Linux_32bit/mongo/mongod --nohttpinterface --noprealloc --bind_ip 127.0.0.1 --smallfiles --port 59544 --dbpath /data/db/repl6tests-2-slave --slave --source 127.0.0.1:33058 --oplogSize 1
m59544| Mon Dec 21 13:21:53 Mongo DB : starting : pid = 2117 port = 59544 dbpath = /data/db/repl6tests-2-slave master = 0 slave = 1 32-bit
m59544|
m59544| ** NOTE: when using MongoDB 32 bit, you are limited to about 2 gigabytes of data
m59544| ** see http://blog.mongodb.org/post/137788967/32-bit-limitations for more
m59544|
m59544| Mon Dec 21 13:21:53 db version v1.3.0-, pdfile version 4.5
m59544| Mon Dec 21 13:21:53 git version: 499cb647b65ca9d1b0cbdc27f72359b30bf2bf39
m59544| Mon Dec 21 13:21:53 sys info: Linux domU-12-31-39-01-70-B4 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 BOOST_LIB_VERSION=1_37
m59544| Mon Dec 21 13:21:53 waiting for connections on port 59544
m59544| Mon Dec 21 13:21:53 listen(): bind() failed errno:98 Address already in use
allocatePorts allocates, opens, then closes (a) port(s). Our assumption is that the os will not utilize the same ports again soon, but in this failure mode one of the allocated ports is reused for a Mongo client before the mongod server for which the port was allocated is started. Either the system is low on available ports, or the port allocation algorithm does not attempt to avoid reuse on short timescales.
Potential solutions:
1) port allocator could leave port open, then consumer of an allocated port would close it immediately before re-opening it as desired
2) all mongod server instances are started before all mongo client instances - not practical for many of the more complicated repl/pairing tests