Change streams use the “sortKey” metadata mechanism to allow mongos to compute the correct merge sort of change notifications on a sharded cluster. The current serialization format for sort key is
{..., $sortKey: {_data: <hex encoded resume token>, _typeBits: <type bits>}}. This is problematic on two fronts. First, this does not correctly conform with the sort key format’s use of empty key names. Second, cluster planning logic provides a single-valued sort key pattern to the AsyncResultsMerger, despite the existence of both the _data and _typeBits fields in the sort key.

In order to ensure that change streams complies with the sort key format used elsewhere, we will change the $sortKey format to {..., $sortKey: {‘’: {_data: <hex encoded resume token>, _typeBits: <type bits>}}}. That is, we will serialize a single-field sort key whose value is a BSON object representation of the resume token.

This change has upgrade/downgrade implications. We believe that 4.2 mongos is implemented in such a way that it will handle the new format correctly. However, the two formats will not compare correctly, so the format change will only take place after the cluster has been fully upgraded to 4.4. This is implemented by adding a new internal, undocumented flag to the aggregate command called use44SortKeys. This flag will be optional and false by default. It is only legal to set to true on a $changeStream aggregate. Setting the flag to true will cause the cursor to use the new sort key format for its lifetime; otherwise, the cursor will continue to use the old sort key format for its lifetime. A 4.4 mongos will always set use44SortKeys:true when opening a change stream. We will remove support for the old sort key format in 4.6 and will remove support for use44SortKeys in 4.8.

A change stream cursor will use the format associated with the FCV in which it was opened for its lifetime, since changing the format mid-stream is incorrect. We will remove support for the old sort key format after branching for 4.6.