[SERVER-33944] Change stream on sharded collection with non-simple default collation is erroneously invalidated upon chunk migration Created: 16/Mar/18  Updated: 27/Oct/23  Resolved: 12/Mar/19

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Ian Boros Assignee: Backlog - Query Team (Inactive)
Resolution: Gone away Votes: 0
Labels: open_todo_in_code
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-39302 Change streams should always use the ... Closed
Related
Assigned Teams:
Query
Operating System: ALL
Participants:
Linked BF Score: 60

 Description   

As background, when a change stream is opened on a collection that doesn't exist, the change stream is invalidated if the collection is created with a collation other than the collation specified on the change stream.

The following steps can lead to a change stream getting erroneously invalidated (say we have a sharded cluster with 2 shards and 1 mongos):
1) Create a collection with a non-simple collation
2) Shard the collection.
3) Insert some data (all of the data goes to the primary shard, shard 0)
3) Create a change stream on that collection with no collation specified.
4) Move a chunk from shard 0 to shard 1
5) Read from the change stream

At step 3 we will establish cursors on all of the shards, using the collection-default collation. On shard 0 this is whatever non-simple collation we created the collection with. Shard 1 doesn't know the collection exists, so its default is the simple collation.

At step 4, the 'createCollection' oplog entry will get applied on shard 1. This means that when shard 1 reads from the change stream, it will see a 'createCollection' with a collation other than the one being used by the change stream. This will trigger an invalidate.

Here's a repro:

(function() {
    load("jstests/libs/collection_drop_recreate.js");  // For assert[Drop|Create]Collection.
    load("jstests/libs/change_stream_util.js");        // For 'ChangeStreamTest'.
 
    const st = new ShardingTest({
        shards: 2,
        mongos: 1,
        rs: {
            nodes: 1,
        },
    });
    const s = st.s;
    let db = s.getDB("test");
 
    print("Disabling balancer...");
    db.adminCommand( { balancerStop: 1 } )
    let cst = new ChangeStreamTest(db);
 
    const caseInsensitive = {locale: "en_US", strength: 2};
 
    let caseInsensitiveCollectionName = "change_stream_case_insensitive";
    assertDropCollection(db, caseInsensitiveCollectionName);
 
    const caseInsensitiveCollection =
          assertCreateCollection(db, caseInsensitiveCollectionName, {collation: caseInsensitive});
 
 
    assert.commandWorked(db.adminCommand({enableSharding: "test"}));
    st.ensurePrimaryShard('test', st.shard0.shardName);
 
    // TODO: Insert some docs (to shard 0)
    for (let i = 0; i < 10; i += 2) {
        assert.writeOK(caseInsensitiveCollection.insert({_id: i, text: "aBc"}));
        assert.writeOK(caseInsensitiveCollection.insert({_id: i + 1, text: "abc"}));
    }
 
    assert.commandWorked(caseInsensitiveCollection.createIndex({_id: "hashed"},
                                                               {collation: {locale: "simple"}}));
    let res = db.adminCommand(
        {shardCollection: caseInsensitiveCollection.getFullName(),
         key: {_id: 'hashed'}, collation: {locale: "simple"}});
    assert.commandWorked(res);
 
    // open CS.
    const implicitCaseInsensitiveStream = cst.startWatchingChanges({
        pipeline: [
            {$changeStream: {}},
            {$match: {"fullDocument.text": "abc"}},
            // Be careful not to use _id in this projection, as startWatchingChanges() will exclude
            // it by default, assuming it is the resume token.
            {$project: {docId: "$documentKey._id"}}
        ],
        collection: caseInsensitiveCollection
    });
    
    // moveChunk to shard 1.
    assert.commandWorked(db.adminCommand({
        moveChunk: caseInsensitiveCollection.getFullName(),
        find: {_id: 1},
        to: st.rs1.getURL(),
        _waitForDelete: false
    }));
 
    // Read from CS.
    cst.assertNextChangesEqual(
        {cursor: implicitCaseInsensitiveStream, expectedChanges: [{docId: 0}, {docId: 1}]});
 
    st.stop();
})();

resmoke.py --suites=no_passthrough repro-file.js

CC charlie.swanson david.storch nicholas.zolnierz



 Comments   
Comment by Ian Boros [ 06/Mar/19 ]

bernard.gorman Looks good! Once SERVER-39302 goes in I guess we can close this as "gone away."

Comment by Ian Boros [ 05/Mar/19 ]

bernard.gorman I believe that's correct. I'm marking this ticket as "depends on" that work.

It would still be nice to test the scenario described above to be sure this actually goes away once SERVER-39302 is done.

Comment by Bernard Gorman [ 04/Mar/19 ]

ian.boros: am I right in thinking that this will go away when we do SERVER-39302 (i.e. never inherit the collection's default collation)?

Generated at Thu Feb 08 04:35:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.