[SERVER-58137] Missing [resolvedViews] field for inserts and commands on time series collections and views Created: 28/Jun/21  Updated: 27/Dec/23

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: 5.0.0-rc0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: George Zhuang (Inactive) Assignee: Backlog - Query Integration
Resolution: Unresolved Votes: 0
Labels: qi-timeseries, read-only-views
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-50286 Indicate if operation is reading from... Closed
Assigned Teams:
Query Integration
Sprint: Query Execution 2021-07-26, QE 2021-08-09, QE 2021-08-23, QE 2021-09-06, QE 2021-09-20, QE 2021-10-04, QE 2021-10-18, QE 2021-11-01, QE 2021-11-15, QE 2021-11-29, QE 2021-12-13, QE 2021-12-27, QE 2022-01-10, QE 2022-01-24
Participants:

 Description   

Description

In SERVER-50286, the `resolvedViews field was added to slow query log entries for views and time series collections.

The current issue is that the `resolvedViews` field is not present on all slow query logs including insert and create logs. 

 Example Time Series Insert Log

{"t":{"$date":"2021-06-28T16:56:57.528-04:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn4263","msg":"Slow query","attr":{"type":"command","ns":"timeDB.timeseriesCollection","appName":"MongoDB Shell","command":{"insert":"timeseriesCollection","ordered":true,"lsid":{"id":{"$uuid":"37b18558-878f-4403-8d50-375c4d4714f0"}},"$db":"timeDB"},"ninserted":1,"keysInserted":0,"numYields":0,"reslen":45,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":1}},"ReplicationStateTransition":{"acquireCount":{"w":1}},"Global":{"acquireCount":{"w":1}},"Database":{"acquireCount":{"w":1}},"Collection":{"acquireCount":{"w":1}},"Mutex":{"acquireCount":{"r":1}}},"flowControl":{"acquireCount":1,"timeAcquiringMicros":1},"storage":{},"remote":"127.0.0.1:63564","protocol":"op_msg","durationMillis":0}} 

Example Time Series Create Log

{"t":{"$date":"2021-06-28T15:44:26.178-04:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn151","msg":"Slow query","attr":{"type":"command","ns":"timeDB.timeseriesCollection","appName":"MongoDB Shell","command":{"create":"timeseriesCollection","timeseries":{"timeField":"tf"},"lsid":{"id":{"$uuid":"a7ce46ff-77ba-41e3-b0c8-adb39b1a7356"}},"$db":"timeDB"},"keysInserted":1,"numYields":0,"reslen":38,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":4}},"ReplicationStateTransition":{"acquireCount":{"w":6}},"Global":{"acquireCount":{"r":4,"w":2}},"Database":{"acquireCount":{"r":4,"w":2}},"Collection":{"acquireCount":{"r":3,"w":2,"W":1}},"Mutex":{"acquireCount":{"r":7}}},"flowControl":{"acquireCount":2,"timeAcquiringMicros":6},"storage":{},"remote":"127.0.0.1:51958","protocol":"op_msg","durationMillis":183}}

Example View Create Log

{"t":{"$date":"2021-06-28T15:59:47.589-04:00"},"s":"I",  "c":"COMMAND",  "id":51803,   "ctx":"conn151","msg":"Slow query","attr":{"type":"command","ns":"smol.managementFeedback","appName":"MongoDB Shell","command":{"create":"managementFeedback","pipeline":[{"$project":{"management":"$feedback.management","department":1}}],"viewOn":"survey","lsid":{"id":{"$uuid":"a7ce46ff-77ba-41e3-b0c8-adb39b1a7356"}},"$db":"smol"},"keysInserted":1,"numYields":0,"reslen":38,"locks":{"ParallelBatchWriterMode":{"acquireCount":{"r":2}},"ReplicationStateTransition":{"acquireCount":{"w":4}},"Global":{"acquireCount":{"r":3,"w":1}},"Database":{"acquireCount":{"r":3,"w":1}},"Collection":{"acquireCount":{"r":2,"w":1,"W":1}},"Mutex":{"acquireCount":{"r":4}}},"flowControl":{"acquireCount":1,"timeAcquiringMicros":1},"storage":{},"remote":"127.0.0.1:51958","protocol":"op_msg","durationMillis":133}}

 

 

 

 



 Comments   
Comment by Kyle Suarez [ 01/Feb/22 ]

Sorry for losing track of this ticket – moving back to the QE backlog.

Comment by George Zhuang (Inactive) [ 15/Jul/21 ]

kyle.suarez, thanks for the create command suggestion! We can use that.

We are attempting to differentiate whether any specific operation is on a timeseries collection, view, or regular collection. Do you have a suggestion for how to differentiate whether an insert operation is on a timeseries collection or regular collection based on the insert slow log (example above in the original description)? 

This is definitely not a blocker but would be a great low-priority nice-to-have.

Comment by Kyle Suarez [ 14/Jul/21 ]

george.zhuang, I just wanted to ping you on the earlier comment. If this is not a blocker, and the information can be gleaned another way, I am inclined to close this as Won't Do.

Comment by Kyle Suarez [ 30/Jun/21 ]

george.zhuang:

  • For the create command, the presence of the viewOn and pipeline fields indicate that a (regular non-materialized) view is being created. The presence of the timeseries document would indicate if the namespace is a timeseries collection.
  • While timeseries collections are implemented using views, they are not truly either normal collections or normal views. There is a special path taken in the storage code to handle timeseries inserts. Perhaps it would be more useful to call these out as being specifically timeseries collections, rather than just views.
Comment by George Zhuang (Inactive) [ 29/Jun/21 ]

louis.williams, all good to change to an improvement.

 

kyle.suarez, the InTel team uses slow query logs to populate the visual Profiler. In Profiler, we would like to classify all entries as from a collection, time series collection, or view, and we are unable to without the resolvedViews field.

If this requires a large refactor, it is ok for us to only have this classification on queries, but it would be a nice to have on other slow query log lines.

Comment by Kyle Suarez [ 28/Jun/21 ]

george.zhuang, I am interested in why you think this field should be present in the two situations you mention.

The purpose of us adding resolvedViews is to aid in the debugging of slow queries, and to make sure that a TSE helping a customer debug knows up-front that a namespace is a view. The two cases you've mentioned weren't envisioned as being a use case in SERVER-50286:

  • Time series collections are special cases and are known to be views, so I feel like the risk here is low.
  • For the create command, the user is literally specifying the view pipeline as part of the command. Is there really any benefit to adding it to the log line?
Comment by Louis Williams [ 28/Jun/21 ]

george.zhuang, I've changed this to an Improvement. Is this causing a bug?

Generated at Thu Feb 08 05:43:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.