[SERVER-79974] Time-series bucket change stream `shardCollection` events translate shardKey fields Created: 14/Aug/23  Updated: 29/Oct/23  Resolved: 08/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.2.0-rc0, 7.0.2, 7.1.0-rc2

Type: Bug Priority: Major - P3
Reporter: Felipe Gasper Assignee: Tommaso Tocci
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
is caused by SERVER-62356 shardCollection for time-series is no... Closed
Related
related to SERVER-80943 Add a test to assert the change strea... Backlog
Assigned Teams:
Sharding EMEA
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.1, v7.0
Sprint: Sharding EMEA 2023-08-21, Sharding EMEA 2023-09-04, Sharding EMEA 2023-09-18
Participants:

 Description   

Problem

Against v7 and latest I see this in a `shardCollection` event:

{operationDescription [{shardKey [{mytimefield 1}]} {unique false} {numInitialChunks 0} {presplitHashedZones false} {timeseries [{timeField mytimefield} {metaField mymetafield} {granularity seconds} {bucketMaxSpanSeconds 3600}]} {collation [{locale simple}]}]}

Note the translated, “logical” field name.

Against v6, though, I see `control.min.mytimefield` and `meta` as the shardKey field names. Apparently that’s correct, per arun.banala@mongodb.com.

Solution & Acceptance Criteria

v6, v7, and latest should all behave as expected.

Impact

This will impact REP-35. I’m going to try to work around it for now by recognizing logical field names & converting them to the bucket field names.



 Comments   
Comment by Githook User [ 08/Sep/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-79974 Time-series bucket change stream `shardCollection` events translate shardKey fields

(cherry picked from commit 54db4ee7305ab97c25c2afeee73189e438476cd4)
Branch: v7.1
https://github.com/mongodb/mongo/commit/70708d87afd0a64fb7290c8759a33de4b14aecd4

Comment by Githook User [ 08/Sep/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-79974 Time-series bucket change stream `shardCollection` events translate shardKey fields

(cherry picked from commit 54db4ee7305ab97c25c2afeee73189e438476cd4)
Branch: v7.0
https://github.com/mongodb/mongo/commit/b4b64a1efdcc2d15f8048d91055430c950c663a3

Comment by Githook User [ 07/Sep/23 ]

Author:

{'name': 'Tommaso Tocci', 'email': 'tommaso.tocci@mongodb.com', 'username': 'toto-dev'}

Message: SERVER-79974 Time-series bucket change stream `shardCollection` events translate shardKey fields
Branch: master
https://github.com/mongodb/mongo/commit/54db4ee7305ab97c25c2afeee73189e438476cd4

Comment by Felipe Gasper [ 18/Aug/23 ]

tommaso.tocci@mongodb.com

use test;
 
db.createCollection("weather", { timeseries: { timeField: "timestamp" } });
 
cs = db.aggregate([{$changeStream: {showExpandedEvents: true, showSystemEvents: true}}]);
 
db.getSiblingDB("admin").runCommand({shardCollection: "test.weather", key: { timestamp: 1 }});
 
cs.next();
cs.next(); // this might hang, depending on if the createIndexes event actually shows

Comment by Felipe Gasper [ 15/Aug/23 ]

Reproduction against v7:

Enterprise [direct: mongos] test> cs = db.aggregate([{$changeStream: {showExpandedEvents: true, showSystemEvents: true}}])
 
Enterprise [direct: mongos] test> cs.next()
{
  _id: {
    _data: '8264DBDA020000000D2B042C0100296E5A100437FE8D29694642198BBC7FA0E564105E463C6F7065726174696F6E54797065003C7368617264436F6C6C656374696F6E00466F7065726174696F6E4465736372697074696F6E00464673686172644B657900461E6D65746164617461002B02006E756E69717565006E1E6E756D496E697469616C4368756E6B7300296E70726573706C69744861736865645A6F6E6573006E4674696D6573657269657300463C74696D654669656C64003C74696D657374616D70003C6D6574614669656C64003C6D65746164617461003C6772616E756C6172697479003C7365636F6E6473001E6275636B65744D61785370616E5365636F6E6473002C1C2000000004',
    _typeBits: Binary(Buffer.from("820004", "hex"), 0)
  },
  operationType: 'shardCollection',
  clusterTime: Timestamp({ t: 1692129794, i: 13 }),
  collectionUUID: new UUID("37fe8d29-6946-4219-8bbc-7fa0e564105e"),
  wallTime: ISODate("2023-08-15T20:03:14.233Z"),
  ns: { db: 'test', coll: 'system.buckets.weather' },
  operationDescription: {
    shardKey: { metadata: 1 },
    unique: false,
    numInitialChunks: Long("0"),
    presplitHashedZones: false,
    timeseries: {
      timeField: 'timestamp',
      metaField: 'metadata',
      granularity: 'seconds',
      bucketMaxSpanSeconds: 3600
    }
  }
}

Generated at Thu Feb 08 06:42:23 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.