[SERVER-60354] Updates should log the original command in system.profile Created: 30/Sep/21  Updated: 02/Feb/24

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Alya Berciu Assignee: Backlog - Query Integration
Resolution: Unresolved Votes: 0
Labels: neweng, qi-query-stats
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Query Integration
Participants:

 Description   

When testing update command routing it would be helpful to be able to obtain the shardVersion from the profiler to confirm that targeted updates are versioned and updates that are sent to every shard are unversioned.



 Comments   
Comment by Kyle Suarez [ 22/Oct/21 ]

david.storch is suggesting that instead of specifically adding the shard version here, we should just profile the full update/delete command in addition to the profiler entry for each individual statement.

Comment by Kyle Suarez [ 20/Oct/21 ]

Thanks for the context arun.banala and alya.berciu, we're going to discuss this ticket once more in the triage meeting with this in mind.

Comment by Arun Banala [ 20/Oct/21 ]

kyle.suarez For most commands, we log the entire input BSON object (which includes generic parameters like shardVersion ) in the system.profile collection. But for updates( and deletes), we override it with only the single update operation of the input object.

For example, a command field in the profiler entry of an insert would look like,

{
	"op" : "insert",
	"ns" : "testDB.system.buckets.testColl",
	"command" : {
	"insert" : "system.buckets.testColl",
	"bypassDocumentValidation" : false,
	"ordered" : true,
	"documents" : [
		{
			"_id" : ObjectId("616ff5e877a73f6357273cb2"),
			"control" : {
				"min" : {
					"time" : ISODate("2021-10-20T10:56:40.366Z")
				},
				"max" : {
					"time" : ISODate("2021-10-20T10:56:40.366Z")
				},
				"version" : 1
			},
			"data" : {
			}
		}
	],
	"shardVersion" : [
		Timestamp(0, 0),
		ObjectId("000000000000000000000000"),
		Timestamp(0, 0)
	],
	"databaseVersion" : {
		"uuid" : UUID("8901ec15-ad95-4db0-8805-9bc52ddbadbc"),
		"timestamp" : Timestamp(1634727388, 14),
		"lastMod" : 1
	},
	"writeConcern" : {
		"w" : "majority",
		"wtimeout" : 0,
		"provenance" : "implicitDefault"
	},
	"lsid" : {
		"id" : UUID("021ff7a5-6baa-47b0-b5a1-9e982a65c28a"),
		"uid" : BinData(0,"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=")
	},
	"$clusterTime" : {
		"clusterTime" : Timestamp(1634727400, 68),
		"signature" : {
			"hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
			"keyId" : NumberLong(0)
		}
	},
	"$configTime" : Timestamp(1634727400, 65),
	"$topologyTime" : Timestamp(1634727387, 16),
	"$client" : {
		"application" : {
			"name" : "MongoDB Shell"
		},
		"driver" : {
			"name" : "MongoDB Internal Client",
			"version" : "5.1.0-alpha-1621-g2b6d233"
		},
		"os" : {
			"type" : "Linux",
			"name" : "Ubuntu",
			"architecture" : "x86_64",
			"version" : "18.04"
		},
		"mongos" : {
			"host" : "arun:20028",
			"client" : "127.0.0.1:41990",
			"version" : "5.1.0-alpha-1621-g2b6d233"
		}
	},
	"$db" : "testDB"
	}
}

Where as the command field in the profiler entry for an update is

{
	"op" : "update",
	"ns" : "testDB.testColl",
	"command" : {
		"q" : {
		},
		"u" : {
			"$inc" : {
				"hostid.b" : 1
			}
		},
		"multi" : true,
		"upsert" : false
	}
}

As you can see for update and delete commands, we seemed to be dropping a lot of useful information that is attached to the input BSON object.

Comment by Alya Berciu [ 19/Oct/21 ]

kyle.suarez unfortunately, this is not the case for updates. This came up when I was testing SERVER-59180 (see this comment) and trying to verify the shardVersion for sharded timeseries updates. The profiler logs updates using a different format which does not include the shardVersion. CC arun.banala

Comment by Kyle Suarez [ 19/Oct/21 ]

alya.berciu, doesn't system.profile already contain this information, since the entire input command gets sent there? CC david.storch

Generated at Thu Feb 08 05:49:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.