-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
Labels:None
-
Query Optimization
-
Fully Compatible
-
QO 2023-06-26, QO 2023-07-10, QO 2023-07-24, QO 2023-08-07, QO 2023-08-21
Currently, running this $changeStreams test on master branch with no changes, fails on the getTelemetry() call with the error, "was not able to re-parse queryStats key when reading queryStats." Despite the original query being coll.aggregate({$changeStream: {}}, {$addFields: {newField: 1}}), the resulting queryShape is
{ cmdNs: { db: "test", coll: "coll" }, command: "aggregate", pipeline: [ { $_internalChangeStreamOplogMatch: { filter: { $and: [ { ts: { $gte: "?timestamp" } }, { fromMigrate: { $not: { $eq: "?bool" } } }, { $or: [ { $and: [ { $or: [ { $and: [ { o.to: { $eq: "?string" } }, { o.renameCollection: { $exists: "?bool" } } ] }, { o.drop: { $eq: "?string" } }, { o.renameCollection: { $eq: "?string" } } ] }, { ns: { $eq: "?string" } }, { op: { $eq: "?string" } } ] }, { $and: [ { $or: [ { o2.type: { $in: "?array<?string>" } }, { o2.migrateChunkToNewShard: { $exists: "?bool" } }, { o2.refineCollectionShardKey: { $exists: "?bool" } }, { o2.reshardBegin: { $exists: "?bool" } }, { o2.reshardCollection: { $exists: "?bool" } }, { o2.reshardDoneCatchUp: { $exists: "?bool" } }, { o2.shardCollection: { $exists: "?bool" } } ] }, { op: { $eq: "?string" } }, { ns: { $regex: "?string" } } ] }, { $and: [ { ns: { $regex: "?string" } }, { $nor: [ { op: { $eq: "?string" } }, { op: { $eq: "?string" } } ] } ] }, { $and: [ { $or: [ { o.drop: { $regex: "?string" } }, { o.renameCollection: { $regex: "?string" } }, { $and: [ { o.to: { $regex: "?string" } }, { o.renameCollection: { $exists: "?bool" } } ] }, { o.create: { $regex: "?string" } }, { o.createIndexes: { $regex: "?string" } }, { o.commitIndexBuild: { $regex: "?string" } }, { o.dropIndexes: { $regex: "?string" } }, { o.collMod: { $regex: "?string" } } ] }, { op: { $eq: "?string" } }, { ns: { $regex: "?string" } } ] }, { $and: [ { $or: [ { o.applyOps: { $elemMatch: { $and: [ { $or: [ { o.create: { $regex: "?string" } }, { o.createIndexes: { $regex: "?string" } } ] }, { ns: { $regex: "?string" } } ] } } }, { o.applyOps.ns: { $regex: "?string" } }, { prevOpTime: { $not: { $eq: "?object" } } } ] }, { op: { $eq: "?string" } }, { o.partialTxn: { $not: { $eq: "?bool" } } }, { o.prepare: { $not: { $eq: "?bool" } } }, { o.applyOps: { $type: "?array<?number>" } } ] }, { $and: [ { o.commitTransaction: { $eq: "?number" } }, { op: { $eq: "?string" } } ] } ] } ] } } }, { $_internalChangeStreamUnwindTransaction: { filter: { $and: [ { $or: [ { $and: [ { $or: [ { $and: [ { o.to: { $regex: "?string" } }, { o.renameCollection: { $exists: "?bool" } } ] }, { o.collMod: { $regex: "?string" } }, { o.commitIndexBuild: { $regex: "?string" } }, { o.create: { $regex: "?string" } }, { o.createIndexes: { $regex: "?string" } }, { o.drop: { $regex: "?string" } }, { o.dropIndexes: { $regex: "?string" } }, { o.renameCollection: { $regex: "?string" } } ] }, { op: { $eq: "?string" } }, { ns: { $regex: "?string" } } ] }, { $and: [ { ns: { $regex: "?string" } }, { $nor: [ { op: { $eq: "?string" } }, { op: { $eq: "?string" } } ] } ] } ] }, { fromMigrate: { $not: { $eq: "?bool" } } } ] } } }, { $_internalChangeStreamTransform: { startAtOperationTime: "?timestamp", fullDocument: "default", fullDocumentBeforeChange: "off" } }, { $_internalChangeStreamCheckInvalidate: {} }, { $_internalChangeStreamCheckResumability: { resumeToken: "?object" } }, { $_internalChangeStreamCheckTopologyChange: {} }, { $_internalChangeStreamHandleTopologyChange: {} }, { $match: { operationType: { $in: "?array<?string>" } } }, { $addFields: { newField: "?number" } } ] }
I pulled and rebase and rebuilt and the problem persists, so it doesn't seem to be the regex bug fix that got merged this week.
So in other words, we need to bug fix tracking query stats for $changeStream and also log that into the collectionType field of the shape.
We should be able to easily track if it's a changeStream collection type by doing, `liteParsedPipeline.hasChangeStream()` where liteParsedPipeline is an existing var inside runAggregate
- is depended on by
-
-
- Closed
-