[SERVER-18430] SECONDARY can not catch up oplog in upgrade progess Created: 12/May/15 Updated: 12/Jun/15 Resolved: 12/Jun/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Admin, Replication |
| Affects Version/s: | 2.6.9 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Jiangcheng Wu | Assignee: | Sam Kleinman (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Operating System: | ALL |
| Participants: |
| Description |
|
Hi, I tried to upgrade my 2.4.11/12 mixed sharded cluster to 2.6.9, but failed as SECONDARY( hidden, priority=0 ) can not catch up oplog. CLUSTER STRUCTURE: PREPAREDNESS: UPGRADE PROGESS:
done with no errors.
look like this member can not contact with other members. log of this memeber
I've attached the full log and filter log of this member, and mgcfg3.avoscloud.com(10.10.17.10) in the log is ARBITER. and mongodb-org-server: I don't know why this happen. May be it is a bug of 2.6.9? |
| Comments |
| Comment by Sam Kleinman (Inactive) [ 12/Jun/15 ] | |||||||||||||||||
|
Thanks for this information. In 2.6, on startup mongod must open each database while starting up to clean up aborted index builds. Just to be clear, the string "IndexRebuilder" refers to the name of the thread doing this work and not to the action its performing. I'm going to go ahead and close this ticket, as the SERVER project is for bugs with MongoDB itself, and questions about performance and operations are probably better addressed to one of our support channels. Regards, | |||||||||||||||||
| Comment by Jiangcheng Wu [ 10/Jun/15 ] | |||||||||||||||||
|
Hi, IndexRebuild progress cost much time, even we have set --noIndexBuildRetry, and the count number of keyword
in start up log is almost equal to database counts. I can not tell is it rebuilding index or just checking index from log. Thank you. | |||||||||||||||||
| Comment by Sam Kleinman (Inactive) [ 09/Jun/15 ] | |||||||||||||||||
|
I'm glad to hear that you've successfully upgraded now. I just want to ask a few questions about your deployment to better understand the difference in start up time?
Thanks, | |||||||||||||||||
| Comment by Jiangcheng Wu [ 03/Jun/15 ] | |||||||||||||||||
|
Hi
We upgraded our production mongo cluster to 2.6.10 a few days ago. Everything looks ok, except the startup time of mongod instance is about triple than 2.4. | |||||||||||||||||
| Comment by Sam Kleinman (Inactive) [ 27/May/15 ] | |||||||||||||||||
|
As I look through these logs and reports, a couple of things stand out at me:
If you could provide answers to these questions it will help us understand what's going on here. Regards, | |||||||||||||||||
| Comment by Jiangcheng Wu [ 22/May/15 ] | |||||||||||||||||
|
Hi, | |||||||||||||||||
| Comment by Jiangcheng Wu [ 12/May/15 ] | |||||||||||||||||
|
add an image of SlaveReplicationInfo(), with weird timestamp finally, the upgrade process was canceled. And mgsh43 downgrade to 2.4.14, this member can catch up with other members.
|