[SERVER-38265] Change Stream Events on sharded clusters are not returned in real time Created: 27/Nov/18  Updated: 27/Oct/23  Resolved: 30/Nov/18

Status: Closed
Project: Core Server
Component/s: Performance, Sharding
Affects Version/s: 4.0.4
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Kurt Agius Assignee: Eric Sedor
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

1. Setup sharding as shown below:

  • Replicaset (shard servers) (rs0)
    • mongo1.example.com:27018, mongo2.example.com:27018, mongo3.example.com:27018 
  • Replicaset (shard servers) (rs1)
    • mongo4.example.com:27018, mongo5.example.com:27018, mongo6.example.com:27018
  • Config servers
    • mongo1.example.com:27019, mongo2.example.com:27019, mongo3.example.com:27019
  • Mongos servers:
    • mongo1.example.com:27017, mongo2.example.com:27017, mongo3.example.com:27017

--- Sharding Status ---
  sharding version: {
   "_id" : 1,
   "minCompatibleVersion" : 5,
   "currentVersion" : 6,
   "clusterId" : ObjectId("5beadc55eb955fb607ecd4e7")
  }
  shards:
        {  "_id" : "rs0db",  "host" : "rs0db/mongo1.example.com:27018,mongo2.example.com:27018,mongo3.example.com:27018",  "state" : 1,  "tags" : [ "rs0zone" ] }
        {  "_id" : "rs1db",  "host" : "rs1db/mongo4.example.com:27018,mongo5.example.com:27018,mongo6.example.com:27018",  "state" : 1,  "tags" : [ "rs1zone" ] }
  active mongoses:
        "4.0.3" : 3
        "4.0.4" : 1
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled:  yes
        Currently running:  no
        Failed balancer rounds in last 5 attempts:  0
        Migration Results for the last 24 hours:
                No recent migrations

2. Run the snippet as shown below:

  • if ran against mongos
    • it will print changes in "batches" (oldest change can be 10seconds delayed, latest change can be 0 seconds delayed)
  • if ran against mongod (shard)
    • it will print changes "live" (delay=0)

var changeStream = db.getMongo().watch();
while(!changeStream.isExhausted()){
 if(changeStream.hasNext()){
  doc = changeStream.next();
  c = new Date().getTime() - new Date(doc.clusterTime.getTime()*1000).getTime();
  c = Math.floor(c/1000).toString();
  print(c); //Print delay for each received change
 }
}

 

 

Participants:

 Description   

When using a sharded setup, trying to tail changestream via mongos (as suggested by the docs), returns changes in "batches" instead of live.

If you tail the changestream via the replicaset directly (which is not the point, because you want to use sharding), then it works as expected i.e updates come in "live" not in "batches".

You can see if an update is coming in "live" or "delayed" by printing the time difference between now & the clusterTime (from the received change).

This is tested this against:

  • mongo client
  • a database with sharding enabled
  • a database without sharding enabled

They all reproduce the same effect.



 Comments   
Comment by Eric Sedor [ 30/Nov/18 ]

Hi Kurt, thanks for your patience.

This is expected for the reasons described in our change stream production recommendations for sharded clusters.

If you'd like to discuss this further, please post on the mongodb-user group or Stack Overflow with the mongodb tag. A question like this involving more discussion would be best posted on the mongodb-user group.

Generated at Thu Feb 08 04:48:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.