Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47863

Initial Sync Progress Metrics

    • Fully Compatible
    • v4.4, v4.2, v4.0, v3.6
    • Repl 2020-10-19, Repl 2020-11-16, Repl 2020-11-30

      Improve the Initial Sync Progress Metrics. 

      Proposed new structure in ReplSetGetStatus:

      New metrics in bold

      "initialSyncStatus": {
           "failedInitialSyncAttempts": 0,
           "maxFailedInitialSyncAttempts": 10,
             *approxTotalDataSize*
             *approxTotalBytesCopied*
           "initialSyncStart": ISODate("2019-12-04T05:12:35.719Z"),
             *totalInitialSyncElapsedMillis*
             *remainingInitialSyncEstimatedMillis*
           "initialSyncAttempts": [],
           "fetchedMissingDocs": 0,
           "appliedOps": 0,
           "initialSyncOplogStart": Timestamp(1575436355, 1),
           "databases": {
                *databasesToClone*
             "databasesCloned": 2,
             "admin": {
               "collections": 4,
               "clonedCollections": 4,
               "start": ISODate("2019-12-04T05:12:35.947Z"),
               "end": ISODate("2019-12-04T05:12:36.486Z"),
               "elapsedMillis": 539,
               "admin.system.roles":{
                  "documentsToCopy": 12,
                  "documentsCopied": 12,
                   *bytesToCopy*
                   *approxBytesCopied*
                  "indexes": 2,
                  "fetchedBatches": 1,
                  "start": ISODate("2019-12-04T05:12:35.950Z"),
                  "end": ISODate("2019-12-04T05:12:36.101Z"),
                  "elapsedMillis": 151,
                  "receivedBatches": 1 
                },
             },
          }
       },
      

       

      New Metrics Details

      • totalInitialSyncElapsedMillis
      • = Current Time - Start Time
      • remainingInitialSyncEstimatedMillis
      • = (totalInitialSyncElapsedMillis / approxTotalBytesCopied) * (approxTotalDataSize - approxTotalBytesCopied)
      • If (approxBytesCopied == 0), this field will not be shown
      • <collection>.bytesToCopy
      • = collStats.size
      • <collection>.approxBytesCopied
      • = collStats.avgObjSize * documentsCopied
      • databasesToClone
      • = length of listDatabases response
      • approxTotalDataSize
      • = Sum (dbStats.dataSize for all databases)
      • approxTotalBytesCopied
      • = Sum (collection.approxBytesCopied for all collections)

      Proposed changes to rs.status()

      • Inherit changes to replSetGetStatus
      • No changes to rs.status() specifically

      Proposed changes to db.getReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printSlaveReplicationInfo()

      Current output:

      When doing Initial Sync:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
      source: localhost:27018
                syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
                0 secs (0 hrs) behind the primary
      source: localhost:27019
                syncedTo: Thu Jan 01 1970 01:00:00 GMT+0100 (CET)
                1587382727 secs (440939.65 hrs) behind the primary
       

      Suggested output:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
      source: localhost:27018
                syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
                0 secs (0 hrs) behind the primary
      source: localhost:27019
                 *InitialSyncSyncSource: <host:port>*
                 *InitialSyncRemainingEstimatedDuration: “remainingInitialSyncEstimatedMillis” in X hours Y minutes*
      

       

            Assignee:
            jason.chan@mongodb.com Jason Chan
            Reporter:
            evin.roesle@mongodb.com Evin Roesle
            Votes:
            3 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved: