[SERVER-36526] There are no change stream events for collection on `mongos` with sharded cluster Created: 08/Aug/18  Updated: 27/Oct/23  Resolved: 14/Aug/18

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.0.1
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Artem Assignee: Nicholas Zolnierz
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Operating System: ALL
Backport Requested:
v4.0
Steps To Reproduce:
  1. Run sharded cluster

$ mongo --nodb
new ShardingTest({ shards: 2, mongos: 1, rs: true })

  1. In another session connect to mongos and run simple script for insert 10 records per second:

$ mongo mongodb://localhost:20009/foo
MongoDB server version: 4.0.1
mongos> for (;;) {db.bar.insert({_id: ObjectId()}); sleep(100);} 

  1. In third session connect to `mongos` and try to create change stream:

$ mongo mongodb://localhost:20009/foo
MongoDB server version: 4.0.1
mongos> db.bar.count()
13116
mongos> db.bar.watch()
mongos> 

On this step I expected see a change stream events, but watch() method wait one second and return empty result.

Sprint: Query 2018-08-13, Query 2018-08-27
Participants:
Case:

 Description   

I can't read change stream events from `mongos` on sharded cluster, but change stream worked on replica set node as expected.

 

Version:

$ mongo --version
MongoDB shell version v4.0.1
git version: 54f1582fc6eb01de4d4c42f26fc133e623f065fb
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64 



 Comments   
Comment by Artem [ 12/Aug/18 ]

This was an extremely useful answer.

This options fix changeStreams on 4.0.1 and help me to fix very similar "no changeStream events issue" in my unit tests on 3.6.6.

I once used the noop-event to synchronize data from oplog of different replicasets. But then I had to give it up: because of strange optimization, noop does not send if there was write event in any other collection. As a result, I received noop only on the test environment.

Comment by Nicholas Zolnierz [ 10/Aug/18 ]

Hi bozaro,

Thanks again for the report. I was able to root cause the problem to a fix for change streams that went into 4.0.1 related to collections that don't have chunks on all shards (SERVER-32088). The unfortunate side effect of that fix is that a $changeStream against an unsharded collection in a sharded cluster will need to wait for a response from the non-primary shard even though it is not aware of the unsharded collection.

The way to get around this is by starting up each mongod with the noop writer. Note that ShardingTest by default will have the noop writer turned off, but otherwise it is on by default with an interval of 10 seconds. 

new ShardingTest({ shards: 2, mongos: 1, rs: {nodes: 1, setParameter: {periodicNoopIntervalSecs: 1, writePeriodicNoops: true}}})

Then you will be able to iterate the cursor returned from the `collection.watch()` shell helper as follows:

let cursor = db.bar.watch();
assert.soon(() => cursor.hasNext());
<cursor.next() will contain the next notification in the stream>

Can you please give that a try and let me know if you still see the issue?

Thanks,
Nick

Comment by David Storch [ 08/Aug/18 ]

Assigning to nicholas.zolnierz to take a first look.

Comment by Nick Brewer [ 08/Aug/18 ]

bozaro Thanks for your report. I've passed this along to our Query team for evaluation.

-Nick

Generated at Thu Feb 08 04:43:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.