Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-47863

Initial Sync Progress Metrics

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.9.0, 4.4.3, 4.2.12
    • Component/s: None
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v4.4, v4.2, v4.0, v3.6
    • Sprint:
      Repl 2020-10-19, Repl 2020-11-16, Repl 2020-11-30

      Description

      Improve the Initial Sync Progress Metrics. 

      Proposed new structure in ReplSetGetStatus:

      New metrics in bold

      "initialSyncStatus": {
           "failedInitialSyncAttempts": 0,
           "maxFailedInitialSyncAttempts": 10,
             *approxTotalDataSize*
             *approxTotalBytesCopied*
           "initialSyncStart": ISODate("2019-12-04T05:12:35.719Z"),
             *totalInitialSyncElapsedMillis*
             *remainingInitialSyncEstimatedMillis*
           "initialSyncAttempts": [],
           "fetchedMissingDocs": 0,
           "appliedOps": 0,
           "initialSyncOplogStart": Timestamp(1575436355, 1),
           "databases": {
                *databasesToClone*
             "databasesCloned": 2,
             "admin": {
               "collections": 4,
               "clonedCollections": 4,
               "start": ISODate("2019-12-04T05:12:35.947Z"),
               "end": ISODate("2019-12-04T05:12:36.486Z"),
               "elapsedMillis": 539,
               "admin.system.roles":{
                  "documentsToCopy": 12,
                  "documentsCopied": 12,
                   *bytesToCopy*
                   *approxBytesCopied*
                  "indexes": 2,
                  "fetchedBatches": 1,
                  "start": ISODate("2019-12-04T05:12:35.950Z"),
                  "end": ISODate("2019-12-04T05:12:36.101Z"),
                  "elapsedMillis": 151,
                  "receivedBatches": 1 
                },
             },
          }
       },
      

       

      New Metrics Details

      • totalInitialSyncElapsedMillis
      • = Current Time - Start Time
      • remainingInitialSyncEstimatedMillis
      • = (totalInitialSyncElapsedMillis / approxTotalBytesCopied) * (approxTotalDataSize - approxTotalBytesCopied)
      • If (approxBytesCopied == 0), this field will not be shown
      • <collection>.bytesToCopy
      • = collStats.size
      • <collection>.approxBytesCopied
      • = collStats.avgObjSize * documentsCopied
      • databasesToClone
      • = length of listDatabases response
      • approxTotalDataSize
      • = Sum (dbStats.dataSize for all databases)
      • approxTotalBytesCopied
      • = Sum (collection.approxBytesCopied for all collections)

      Proposed changes to rs.status()

      • Inherit changes to replSetGetStatus
      • No changes to rs.status() specifically

      Proposed changes to db.getReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printReplicationInfo()

      • No change, the current metrics are unrelated to initial sync

      Proposed changes to rs.printSlaveReplicationInfo()

      Current output:

      When doing Initial Sync:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
      source: localhost:27018
                syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
                0 secs (0 hrs) behind the primary
      source: localhost:27019
                syncedTo: Thu Jan 01 1970 01:00:00 GMT+0100 (CET)
                1587382727 secs (440939.65 hrs) behind the primary
       

      Suggested output:

      MongoDB Enterprise replset:PRIMARY> rs.printSlaveReplicationInfo()
      source: localhost:27018
                syncedTo: Mon Apr 20 2020 12:38:47 GMT+0100 (WEST)
                0 secs (0 hrs) behind the primary
      source: localhost:27019
                 *InitialSyncSyncSource: <host:port>*
                 *InitialSyncRemainingEstimatedDuration: “remainingInitialSyncEstimatedMillis” in X hours Y minutes*
      

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jason.chan Jason Chan
              Reporter:
              evin.roesle Evin Roesle (Inactive)
              Participants:
              Votes:
              3 Vote for this issue
              Watchers:
              21 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: