Initial Sync Progress Metrics

XMLWordPrintableJSON

    • Fully Compatible
    • v4.4, v4.2, v4.0, v3.6
    • Repl 2020-10-19, Repl 2020-11-16, Repl 2020-11-30
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Improve the Initial Sync Progress Metrics. 

      Proposed new structure in ReplSetGetStatus:

      New metrics in bold

      "initialSyncStatus": {
     "failedInitialSyncAttempts": 0,
     "maxFailedInitialSyncAttempts": 10,
       *approxTotalDataSize*
       *approxTotalBytesCopied*
     "initialSyncStart": ISODate("2019-12-04T05:12:35.719Z"),
       *totalInitialSyncElapsedMillis*
       *remainingInitialSyncEstimatedMillis*
     "initialSyncAttempts": [],
     "fetchedMissingDocs": 0,
     "appliedOps": 0,
     "initialSyncOplogStart": Timestamp(1575436355, 1),
     "databases": {
          *databasesToClone*
       "databasesCloned": 2,
       "admin": {
         "collections": 4,
         "clonedCollections": 4,
         "start": ISODate("2019-12-04T05:12:35.947Z"),
         "end": ISODate("2019-12-04T05:12:36.486Z"),
         "elapsedMillis": 539,
         "admin.system.roles":{
            "documentsToCopy": 12,
            "documentsCopied": 12,
             *bytesToCopy*
             *approxBytesCopied*
            "indexes": 2,
            "fetchedBatches": 1,
            "start": ISODate("2019-12-04T05:12:35.950Z"),
            "end": ISODate("2019-12-04T05:12:36.101Z"),
            "elapsedMillis": 151,
            "receivedBatches": 1 
          },
       },
    }
 },

       

      New Metrics Details

      • totalInitialSyncElapsedMillis
      • = Current Time - Start Time
      • remainingInitialSyncEstimatedMillis
      • = (totalInitialSyncElapsedMillis / approxTotalBytesCopied) * (approxTotalDataSize - approxTotalBytesCopied)
      • If (approxBytesCopied == 0), this field will not be shown
      • <collection>.bytesToCopy
      • = collStats.size
      • <collection>.approxBytesCopied
      • = collStats.avgObjSize * documentsCopied
      • databasesToClone
      • = length of listDatabases response
      • approxTotalDataSize
      • = Sum (dbStats.dataSize for all databases)
      • approxTotalBytesCopied
      • = Sum (collection.approxBytesCopied for all collections)

      Proposed changes to rs.status()

      • Inherit changes to replSetGetStatus
      • No changes to rs.status() specifically

      Proposed changes to db.getReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printSlaveReplicationInfo()

      Current output:

      When doing Initial Sync:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
source: localhost:27018
          syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
          0 secs (0 hrs) behind the primary
source: localhost:27019
          syncedTo: Thu Jan 01 1970 01:00:00 GMT+0100 (CET)
          1587382727 secs (440939.65 hrs) behind the primary
 

      Suggested output:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
source: localhost:27018
          syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
          0 secs (0 hrs) behind the primary
source: localhost:27019
           *InitialSyncSyncSource: <host:port>*
           *InitialSyncRemainingEstimatedDuration: “remainingInitialSyncEstimatedMillis” in X hours Y minutes*

       

            Assignee:
            Jason Chan
            Reporter:
            Evin Roesle
            Votes:
            3 Vote for this issue
            Watchers:
            22 Start watching this issue

              Created:
              Updated:
              Resolved: