[SERVER-19566] Profiler format changes for the find/getMore commands Created: 23/Jul/15  Updated: 22/Mar/17  Resolved: 03/Sep/15

Status: Closed
Project: Core Server
Component/s: Diagnostics, Querying
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Improvement Priority: Major - P3
Reporter: J Rassi Assignee: David Storch
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-6141 Doc changes to the profiler format fo... Closed
Related
related to SERVER-19567 Compatibility layer for display of fi... Closed
is related to CXX-698 DBClientTest.Comment uses incorrect q... Closed
Backwards Compatibility: Major Change
Sprint: Quint 9 09/18/15
Participants:

 Description   

The profile entry format should not depend on whether or not the query was delivered as an OP_QUERY wire protocol find, or via the find/getMore commands. We plan to upconvert profile data collected for OP_QUERY find in order to look more like the find command. This will involve the following changes to the profile entry schema:

  • The query field will contain a documented formatted like the find command parameters for both actual find commands and OP_QUERY find. For instance, the query field might look something like {find: "collection", filter: {a: {$gt: 0}}, hint: {a: 1}, returnKey: true}.
  • The nscanned field will be renamed to keysExamined, for consistency with explain.
  • The nscannedObjects field will be renamed to docsExamined, for consistency with explain.
  • The scanAndOrder field will be renamed to hasSortStage, for consistency with explain.
  • The ntoskip field will be removed. This information will now be available in the field query.skip, since skip is a find command parameter.
  • The ntoreturn field will similarly be removed. For OP_QUERY, it will instead be present in query.ntoreturn. For the find command, which has true limit and batchSize fields, the comparable information will be available in query.limit and query.batchSize.

See below for a few entries of profiler entries with the new format:

{ "op" : "query", "ns" : "test.c", "query" : { "find" : "c", "filter" : { "a" : 1 } }, "keysExamined" : 2, "docsExamined" : 2, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 2, "responseLength" : 108, "millis" : 0, "execStats" : { "stage" : "FETCH", "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 3, "advanced" : 2, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 2, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 3, "advanced" : 2, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "keyPattern" : { "a" : 1 }, "indexName" : "a_1", "isMultiKey" : false, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "a" : [ "[1.0, 1.0]" ] }, "keysExamined" : 2, "dupsTested" : 0, "dupsDropped" : 0, "seenInvalidated" : 0 } }, "ts" : ISODate("2015-09-03T15:26:14.948Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.c", "query" : { "find" : "c", "filter" : { "a" : 1 }, "sort" : { "b" : 1 } }, "keysExamined" : 6, "docsExamined" : 6, "hasSortStage" : true, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 2, "responseLength" : 108, "millis" : 1, "execStats" : { "stage" : "FETCH", "filter" : { "a" : { "$eq" : 1 } }, "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 6, "advanced" : 2, "needTime" : 2, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 4, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 4, "executionTimeMillisEstimate" : 0, "works" : 5, "advanced" : 4, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "keyPattern" : { "b" : 1 }, "indexName" : "b_1", "isMultiKey" : false, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "b" : [ "[MinKey, MaxKey]" ] }, "keysExamined" : 4, "dupsTested" : 0, "dupsDropped" : 0, "seenInvalidated" : 0 } }, "ts" : ISODate("2015-09-03T15:26:21.196Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.c", "query" : { "find" : "c", "ntoreturn" : NumberLong(4) }, "cursorid" : 22422431767, "keysExamined" : 0, "docsExamined" : 4, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 4, "responseLength" : 196, "millis" : 0, "execStats" : { "stage" : "COLLSCAN", "filter" : { "$and" : [ ] }, "nReturned" : 4, "executionTimeMillisEstimate" : 0, "works" : 5, "advanced" : 4, "needTime" : 1, "needYield" : 0, "saveState" : 1, "restoreState" : 0, "isEOF" : 0, "invalidates" : 0, "direction" : "forward", "docsExamined" : 4 }, "ts" : ISODate("2015-09-03T15:26:30.340Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.c", "query" : { "find" : "c", "skip" : NumberLong(2), "ntoreturn" : NumberLong(4) }, "keysExamined" : 0, "docsExamined" : 4, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 2, "responseLength" : 108, "millis" : 0, "execStats" : { "stage" : "SKIP", "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 6, "advanced" : 2, "needTime" : 3, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "skipAmount" : 0, "inputStage" : { "stage" : "COLLSCAN", "filter" : { "$and" : [ ] }, "nReturned" : 4, "executionTimeMillisEstimate" : 0, "works" : 6, "advanced" : 4, "needTime" : 1, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "direction" : "forward", "docsExamined" : 4 } }, "ts" : ISODate("2015-09-03T15:26:34.571Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }
{ "op" : "query", "ns" : "test.c", "query" : { "find" : "c", "filter" : {  }, "skip" : NumberLong(2), "ntoreturn" : NumberLong(4), "hint" : { "a" : 1 } }, "keysExamined" : 4, "docsExamined" : 4, "cursorExhausted" : true, "keyUpdates" : 0, "writeConflicts" : 0, "numYield" : 0, "locks" : { "Global" : { "acquireCount" : { "r" : NumberLong(2) } }, "Database" : { "acquireCount" : { "r" : NumberLong(1) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(1) } } }, "nreturned" : 2, "responseLength" : 108, "millis" : 0, "execStats" : { "stage" : "SKIP", "nReturned" : 2, "executionTimeMillisEstimate" : 0, "works" : 5, "advanced" : 2, "needTime" : 2, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "skipAmount" : 0, "inputStage" : { "stage" : "FETCH", "nReturned" : 4, "executionTimeMillisEstimate" : 0, "works" : 5, "advanced" : 4, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "docsExamined" : 4, "alreadyHasObj" : 0, "inputStage" : { "stage" : "IXSCAN", "nReturned" : 4, "executionTimeMillisEstimate" : 0, "works" : 5, "advanced" : 4, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "keyPattern" : { "a" : 1 }, "indexName" : "a_1", "isMultiKey" : false, "isUnique" : false, "isSparse" : false, "isPartial" : false, "indexVersion" : 1, "direction" : "forward", "indexBounds" : { "a" : [ "[MinKey, MaxKey]" ] }, "keysExamined" : 4, "dupsTested" : 0, "dupsDropped" : 0, "seenInvalidated" : 0 } } }, "ts" : ISODate("2015-09-03T15:26:39.140Z"), "client" : "127.0.0.1", "allUsers" : [ ], "user" : "" }



 Comments   
Comment by Githook User [ 03/Sep/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-19566 convert OP_QUERY profiler entries to look like the find command

Includes the following changes to profiler entries for queries:

– The query field will contain find command parameters, e.g.
{find: <coll>, filter: {...}}.
– Renames 'nscanned' to 'keysExamined' and 'nscannedObjects' to 'docsExamined'.
– Renames 'scanAndOrder' to 'hasSortStage'.
– Removes the top-level ntoreturn and ntoskip fields. Skip, limit, and
batchSize info will be present in the 'query' field instead.
Branch: master
https://github.com/mongodb/mongo/commit/1574457c5ff972f39ddd079164be894ae6fc9f59

Comment by Githook User [ 28/Jul/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-19566 disable tests
Branch: master
https://github.com/mongodb/mongo/commit/f13c593b399718cde59f096a65b3f2131f067f16

Comment by David Storch [ 27/Jul/15 ]

As part of the work for this ticket, tests disabled in https://github.com/mongodb/mongo/commit/f13c593b399718cde59f096a65b3f2131f067f16 should be re-enabled.

Comment by Githook User [ 27/Jul/15 ]

Author:

{u'username': u'dstorch', u'name': u'David Storch', u'email': u'david.storch@10gen.com'}

Message: SERVER-19566 disable tests
Branch: master
https://github.com/mongodb/mongo/commit/f13c593b399718cde59f096a65b3f2131f067f16

Generated at Thu Feb 08 03:51:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.