Initial Sync Performance

XMLWordPrintableJSON

    • Type: Question
    • Resolution: Won't Fix
    • Priority: Major - P3
    • None
    • Affects Version/s: 2.2.1, 2.4.6
    • Component/s: Replication
    • None
    • Environment:
      OpenSuse x64
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      After multiple harddisk failures on the secondary member of a replica set we are desperatly trying bring a new member up and running without taking down the whole system and manually copying the files from the primary

      after mongo completed copying the data, the sync process is more or less stuck on creating the indices.
      this process already took almost 1 week! and is still not finished yet

      any error or loss of connection to the primary in the sync process will restart the whole process from the beginning, deleting all previously synced data and indices

      log excerpt:

      2013-09-20T11:22:07.010835+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:07.010 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141451300/3764787709#01183%
      2013-09-20T11:22:17.005409+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:17.004 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141577600/3764787709#01183%
      2013-09-20T11:22:27.010042+02:00 rack4-5 mongod.30000[25590]: Fri Sep 20 11:22:27.009 [rsSync] #011#011Index: (2/3) BTree Bottom Up Progress: 3141705300/3764787709#01183%

      Size of the Data is currently at 2.4TB, increasing at a rate of about 20GB per day

      is there any way to speed up this process?
      is there any way to have some sort of incremental initial sync,since in case of an error, having to restart the whole process from the beginning is utterly insane
      why do indices have to be rebuilt from scratch instead of just getting copied from the primary anyway

      currently mongodb is basically unusable for our purpose, because in the event of a hardware failure, we are unable to get a replacement back online in a reasonable amount of time without shutting down the whole database

            Assignee:
            Unassigned
            Reporter:
            Winand Appelhoff
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: