Hi, I tried to upgrade my 2.4.11/12 mixed sharded cluster to 2.6.9, but failed as SECONDARY( hidden, priority=0 ) can not catch up oplog.
3 SHARDS, each mongodb instances listened on port 27018
3 configservers, each config server listened on port 27019
lots mongos deployed on app servers and listened on port 27020
As guide http://docs.mongodb.org/manual/release-notes/2.6-upgrade/#preparedness said, we did some checks and found some problems descirbed on https://jira.mongodb.org/browse/SERVER-17746 .
And we added args --setParameter failIndexKeyTooLong=false to avoid write failure , and ignored other errors .
As guide http://docs.mongodb.org/manual/release-notes/2.6-upgrade/#upgrade-a-sharded-cluster-to-2-6 said.
1, disabled write operations to metadata, done with no errors
2, disabled balancer, done with no errors
3, upgrade metad data, done with no errors
4, upgrade all mongos instanced to 2.6.9, done with no errors
5, stopped all 3 ARBITER role of each SHARD ( 3 in total ), as ARIBITER and mongoconfig running on the same box and intall mongodb-org-server(2.6.9) will remove package mongodb-10gen(2.4.x).
6, upgrade all 3 mongoconfig servers in reverse sequence of
leaving the first system in the mongos --configdb argument to upgrade last.
done with no errors.
7, picked a HIDDEN MEMBER of a SHARD, tried to upgrade and failed as SECONARY can not catch up oplog.
and other PRIMARY and SECONDARY roles shows:
look like this member can not contact with other members.
log of this memeber
I've attached the full log and filter log of this member, and mgcfg3.avoscloud.com(10.10.17.10) in the log is ARBITER.
db version v2.6.9
git version: df313bc75aa94d192330cb92756fc486ea604e64
I don't know why this happen. May be it is a bug of 2.6.9?