[KAFKA-296] Confused with ADD_ALT_NAMESPACE_STAGE and UNSET_ORIGINAL_NAMESPACE_STAGE in CopyData Created: 24/Feb/22  Updated: 27/Oct/23  Resolved: 15/Mar/22

Status: Closed
Project: Kafka Connector
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Question Priority: Trivial - P5
Reporter: Jiabao Sun Assignee: Ross Lawley
Resolution: Works as Designed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Hi there,

 
I am a little confused with the pipeline stages of ADD_ALT_NAMESPACE_STAGE and UNSET_ORIGINAL_NAMESPACE_STAGE in MongoCopyDataManager.
May I ask the reason we need these two stages?
 

private static final String PIPELINE_TEMPLATE =
    format(
        "{$replaceRoot: "
            + "{newRoot: {"
            + "_id: {_id: '$_id', copyingData: true}, "
            + "operationType: 'insert', "
            + "%s: {db: '%%s', coll: '%%s'}"
            + "documentKey: {_id: '$_id'}, "
            + "fullDocument: '$$ROOT'}}"
            + "}",
        NAMESPACE_FIELD);
 
private static final BsonDocument ADD_ALT_NAMESPACE_STAGE =
    BsonDocument.parse(
        format("{'$addFields': {'%s': '$%s'}}", ALT_NAMESPACE_FIELD, NAMESPACE_FIELD));
private static final BsonDocument UNSET_ORIGINAL_NAMESPACE_STAGE =
    BsonDocument.parse(format("{'$project': {'%s': 0}}", NAMESPACE_FIELD));

 
 



 Comments   
Comment by Ross Lawley [ 15/Mar/22 ]

I replied via email on the same subject.

Those pipeline stages were added to work around a limitation for some cloud provider users. When using your own MongoDB cluster then there would be no need for the workaround.

Generated at Thu Feb 08 09:06:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.