Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10833

Initial Sync Performance

    XMLWordPrintableJSON

Details

    • Icon: Question Question
    • Resolution: Won't Fix
    • Icon: Major - P3 Major - P3
    • None
    • 2.2.1, 2.4.6
    • Replication
    • None
    • OpenSuse x64

    Description

      After multiple harddisk failures on the secondary member of a replica set we are desperatly trying bring a new member up and running without taking down the whole system and manually copying the files from the primary

      after mongo completed copying the data, the sync process is more or less stuck on creating the indices.
      this process already took almost 1 week! and is still not finished yet

      any error or loss of connection to the primary in the sync process will restart the whole process from the beginning, deleting all previously synced data and indices

      log excerpt:

      2013-09-20T11:22:07.010835+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:07.010 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141451300/3764787709#01183%
      2013-09-20T11:22:17.005409+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:17.004 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141577600/3764787709#01183%
      2013-09-20T11:22:27.010042+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:27.009 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141705300/3764787709#01183%

      Size of the Data is currently at 2.4TB, increasing at a rate of about 20GB per day

      is there any way to speed up this process?
      is there any way to have some sort of incremental initial sync,since in case of an error, having to restart the whole process from the beginning is utterly insane
      why do indices have to be rebuilt from scratch instead of just getting copied from the primary anyway

      currently mongodb is basically unusable for our purpose, because in the event of a hardware failure, we are unable to get a replacement back online in a reasonable amount of time without shutting down the whole database

      Attachments

        Activity

          People

            Unassigned Unassigned
            appelhoff@kajomi.de Winand Appelhoff
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: