[SERVER-29490] mongodb secondary keep lagging Created: 07/Jun/17  Updated: 07/Jun/17  Resolved: 07/Jun/17

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.2.14
Fix Version/s: None

Type: Question Priority: Major - P3
Reporter: Wade [X] Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu 1404


Participants:

 Description   

Mongodb secondary keep lag of time – The total count is not the same. but the latest 10 records are the same.

financeset:PRIMARY> db.user_account.count()
694053
financeset:PRIMARY> new Date()
ISODate("2017-06-07T10:37:09.718Z")

financeset:SECONDARY> db.user_account.count()
694047
financeset:SECONDARY> new Date()
ISODate("2017-06-07T10:37:19.889Z")

financeset:PRIMARY> new Date()
ISODate("2017-06-07T10:38:56.093Z")
financeset:PRIMARY> db.user_account.count()
694054

financeset:SECONDARY> db.user_account.count()
694048

And PRIMARY user_account - SECONDARY(puma) user_account = 6 (for a quite long period, always around 5~6)

financeset:SECONDARY> db.user_account.find({},{_id:-1}).limit(8)

{ "_id" : ObjectId("54ce4fac79337f164b365040") } { "_id" : ObjectId("54f2f90bf4db4b5013e63a1d") } { "_id" : ObjectId("54ceede36f48531b4bd2760a") } { "_id" : ObjectId("54cef8d079337f164b36505a") } { "_id" : ObjectId("54cf84cfe8f067d8cea97991") } { "_id" : ObjectId("54cfa6fe489ba090b7a77c04") } { "_id" : ObjectId("54cfb330489ba090b7a77c07") } { "_id" : ObjectId("54d03dab279657ea64b62c4f") }

financeset:PRIMARY> db.user_account.find({},{_id:-1}).limit(8)

{ "_id" : ObjectId("54ce4fac79337f164b365040") } { "_id" : ObjectId("54f2f90bf4db4b5013e63a1d") } { "_id" : ObjectId("54ceede36f48531b4bd2760a") } { "_id" : ObjectId("54cef8d079337f164b36505a") } { "_id" : ObjectId("54cf84cfe8f067d8cea97991") } { "_id" : ObjectId("54cfa6fe489ba090b7a77c04") } { "_id" : ObjectId("54cfb330489ba090b7a77c07") } { "_id" : ObjectId("54d03dab279657ea64b62c4f") }

Also , another secondary has similar phenomenon.

financeset:PRIMARY> db.printSlaveReplicationInfo()
source: gentoo.local:37017
syncedTo: Wed Jun 07 2017 18:37:27 GMT+0800 (CST)
2 secs (0 hrs) behind the primary
source: puma.local:37017
syncedTo: Wed Jun 07 2017 18:37:27 GMT+0800 (CST)
2 secs (0 hrs) behind the primary
source: dolphin.local:37017
syncedTo: Wed Jun 07 2017 17:39:09 GMT+0800 (CST)
3500 secs (0.97 hrs) behind the primary
source: 121.32.55.xxx:37017
syncedTo: Wed Jun 07 2017 17:37:29 GMT+0800 (CST)
3600 secs (1 hrs) behind the primary


According to Ref blog: https://blog.mlab.com/2013/03/replication-lag-the-facts-of-life/#What_is_replication_lag
And https://docs.mongodb.com/manual/reference/program/mongod/#cmdoption-autoresync
Since we’re not master slave , we don’t think the --autoresync option help.

ENV info

#puma is secondary
kkk@puma:~$ free -m
total used free shared buff/cache available
Mem: 96670 39936 575 35 56158 56474
Swap: 65523 6536 58987

The env is local network. Cpu & network & disk is mostly idle.

kkk@tiger:~$ mongod --version
db version v3.2.10
kkk@puma:~$ mongod --version
db version v3.2.10
build environment:
distmod: ubuntu1404
distarch: x86_64
target_arch: x86_64

We don’t see weird log in secondary log.



 Comments   
Comment by Kelsey Schubert [ 07/Jun/17 ]

Hi Wade,

Thanks for your report. Please note that SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-users group.

Kind regards,
Thomas

Generated at Thu Feb 08 04:21:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.