Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-29490

mongodb secondary keep lagging

    XMLWordPrintableJSON

Details

    • Icon: Question Question
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • 3.2.14
    • Replication
    • None
    • ubuntu 1404

    Description

      Mongodb secondary keep lag of time – The total count is not the same. but the latest 10 records are the same.

      financeset:PRIMARY> db.user_account.count()
      694053
      financeset:PRIMARY> new Date()
      ISODate("2017-06-07T10:37:09.718Z")

      financeset:SECONDARY> db.user_account.count()
      694047
      financeset:SECONDARY> new Date()
      ISODate("2017-06-07T10:37:19.889Z")

      financeset:PRIMARY> new Date()
      ISODate("2017-06-07T10:38:56.093Z")
      financeset:PRIMARY> db.user_account.count()
      694054

      financeset:SECONDARY> db.user_account.count()
      694048

      And PRIMARY user_account - SECONDARY(puma) user_account = 6 (for a quite long period, always around 5~6)

      financeset:SECONDARY> db.user_account.find({},{_id:-1}).limit(8)

      { "_id" : ObjectId("54ce4fac79337f164b365040") } { "_id" : ObjectId("54f2f90bf4db4b5013e63a1d") } { "_id" : ObjectId("54ceede36f48531b4bd2760a") } { "_id" : ObjectId("54cef8d079337f164b36505a") } { "_id" : ObjectId("54cf84cfe8f067d8cea97991") } { "_id" : ObjectId("54cfa6fe489ba090b7a77c04") } { "_id" : ObjectId("54cfb330489ba090b7a77c07") } { "_id" : ObjectId("54d03dab279657ea64b62c4f") }

      financeset:PRIMARY> db.user_account.find({},{_id:-1}).limit(8)

      { "_id" : ObjectId("54ce4fac79337f164b365040") } { "_id" : ObjectId("54f2f90bf4db4b5013e63a1d") } { "_id" : ObjectId("54ceede36f48531b4bd2760a") } { "_id" : ObjectId("54cef8d079337f164b36505a") } { "_id" : ObjectId("54cf84cfe8f067d8cea97991") } { "_id" : ObjectId("54cfa6fe489ba090b7a77c04") } { "_id" : ObjectId("54cfb330489ba090b7a77c07") } { "_id" : ObjectId("54d03dab279657ea64b62c4f") }

      Also , another secondary has similar phenomenon.

      financeset:PRIMARY> db.printSlaveReplicationInfo()
      source: gentoo.local:37017
      syncedTo: Wed Jun 07 2017 18:37:27 GMT+0800 (CST)
      2 secs (0 hrs) behind the primary
      source: puma.local:37017
      syncedTo: Wed Jun 07 2017 18:37:27 GMT+0800 (CST)
      2 secs (0 hrs) behind the primary
      source: dolphin.local:37017
      syncedTo: Wed Jun 07 2017 17:39:09 GMT+0800 (CST)
      3500 secs (0.97 hrs) behind the primary
      source: 121.32.55.xxx:37017
      syncedTo: Wed Jun 07 2017 17:37:29 GMT+0800 (CST)
      3600 secs (1 hrs) behind the primary


      According to Ref blog: https://blog.mlab.com/2013/03/replication-lag-the-facts-of-life/#What_is_replication_lag
      And https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-autoresync
      Since we’re not master slave , we don’t think the --autoresync option help.

      ENV info

      #puma is secondary
      kkk@puma:~$ free -m
      total used free shared buff/cache available
      Mem: 96670 39936 575 35 56158 56474
      Swap: 65523 6536 58987

      The env is local network. Cpu & network & disk is mostly idle.

      kkk@tiger:~$ mongod --version
      db version v3.2.10
      kkk@puma:~$ mongod --version
      db version v3.2.10
      build environment:
      distmod: ubuntu1404
      distarch: x86_64
      target_arch: x86_64

      We don’t see weird log in secondary log.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Wade Wade [X]
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: