-
Type:
Investigation
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Tools and Replicator
-
8
Change stream events that originate from a chunk migration (i.e. the internal movement of data between shards) can now carry a new top-level field fromMigrate: true. This field is present on the change stream event itself, not just internally in the oplog, so consumers can distinguish "real" writes from migration-induced writes.
The new fromMigrate field is emitted if both of the following conditions are met:
- The change stream is opened with showMigrationEvents: true. This flag already existed and is what makes migration events visible in the first place. Without it, migration events are suppressed entirely and the new field is not emitted.
Note: showMigrationEvents: true is only accepted on a direct shard mongod connection; mongos (still) rejects it with error 31123. - The server parameter changeStreamsEmitFromMigrate is enabled. This is a new setParameter-settable flag (configurable at startup or runtime), and it defaults to true. Disabling it suppresses the fromMigrate field while still allowing migration events to appear (i.e. you get the visibility of showMigrationEvents without the tagging).
This server parameter is an escape hatch for users that open change streams with showMigrationEvents: true, but do not want the fromMigrate fields to be emitted in change events.
Note: changing the server parameter will only have an effect for change streams opened afterwards. Change streams that are already in flight will not change their behavior when the server parameter is toggled.
The following change events will have fromMigrate: true if the above conditions are met:
- insert and delete events triggered by a moveChunk command.
The inserts will be tagged on the recipient shard, the delete events will be tagged on the donor shard. - create and createIndexes events triggered by moveCollection, movePrimary and unshardCollection commands.
The events will be tagged on the recipient shard. - reshardDoneCatchUp and rename events triggered by moveCollection, unshardCollection and reshardCollection commands.
The events will be tagged on the recipient shard.
The following change events will not have fromMigrate: true even if the above conditions are met:
- any events not originating from a migration.
- insert events triggered by moveCollection, unshardCollection and reshardCollection commands.
These inserts will be written to system.resharding.* temp namespaces, and are owned documents, not orphans.
Note: the only value emitted for the fromMigrate field is true. No events will be emitted with fromMigrate: false.
Backwards-compatibility:
When upgrading to 9.0, change streams opened with showMigrationEvents: true will start emitting the fromMigrate: true for events originating from a migration, but only on nodes that are running v9.0 or higher. Nodes that are not yet running v9.0 or higher will not emit the field yet.
This only affects change streams opened with showMigrationEvents: true.
If this is a concern, users can either ignore the fromMigrate field until all nodes have been upgraded to v9.0 or higher, or set the changeStreamsEmitFromMigrate server parameter on the nodes until all nodes have been upgraded.
Description of Linked Ticket
The undocumented {showMigrationEvents: true} option to $changeStreams causes oplog entries with {fromMigrate: true} to be returned. This behavior was initially introduced for Atlas Search as part of SERVER-38682. One idea for detecting when mongosync would be impacted by REP-4188 was to open a {showMigrationEvents: true} change stream and detect the range deleter running from its delete change events. Unfortunately the {fromMigrate: true} designation is not preserved between the oplog entry and the change event and so there is no way to distinguish application client deletes from range deleter deletes. While a different approach will ultimately be taken for mongosync, we should improve the behavior for the {showMigrationEvents: true} option such that it preserves more information from the underlying oplog entry.
- depends on
-
SERVER-88167 Expose information about change event being from a migration event when change streams run with showMigrationEvents:true
-
- Closed
-