[COMPASS-3157] Enforce maxTimeMS for every query sent to the server Created: 28/Sep/18  Updated: 20/May/19  Resolved: 09/Apr/19

Status: Closed
Project: Compass
Component/s: CRUD, Query, Schema
Affects Version/s: 1.16.0-beta.7, 1.15.4
Fix Version/s: 1.19.0

Type: Story Priority: Major - P3
Reporter: Emilio Scalise Assignee: Durran Jordan
Resolution: Done Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image-2019-05-16-09-52-37-178.png    
Issue Links:
Documented
Related
is related to COMPASS-2546 When schema analysis agg pipeline tim... Closed
Epic Link: COMPASS-2232
Story Points: 3
Documentation Changes: Needed
Documentation Changes Summary:

The querybar now contains a new box to specify maxTimeMS (see attachment)

Sprint: Iteration Antelope, Iteration Bison

 Description   

Currently query filters that can be entered in the "Documents" panel don't use the maxtimems parameter. It is possible with non selective filters not supported by indexes to cause to run very heavy queries (COLLSCAN) on the MongoDB server causing possibly performance issues for the other clients. The parameter maxTimeMS is not set in the count and in the find commands sent.

For example I loaded 5.7M tweets on a test MongoDB instances and I tried to use the filter

{source: "a"}

which causes a full collection scan over 5.7M documents:

2018-09-28T10:42:42.588+0200 I NETWORK  [conn6] received client metadata from 127.0.0.1:51337 conn6: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.6.6" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "17.7.0" } }
2018-09-28T10:44:52.172+0200 I COMMAND  [conn5] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:47574 reslen:44 locks:{ Global: { acquireCount: { r: 95150 } }, Database: { acquireCount: { r: 47575 } }, Collection: { acquireCount: { r: 47575 } } } protocol:op_query 137667ms
2018-09-28T10:46:53.077+0200 I COMMAND  [conn1] command test.tweets appName: "MongoDB Compass Beta" command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47243 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 94488 } }, Database: { acquireCount: { r: 47244 } }, Collection: { acquireCount: { r: 47244 } } } protocol:op_query 120877ms

The query is limited to 20 documents but has no maxTimeMS set.

If you use the schema tab and you run the same filter, maxTimeMS is used, but I noticed that the same queries are then run without the timeout set:

2018-09-28T13:33:44.380+0200 I COMMAND  [conn10] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("bbda3184-e246-44bd-a5dd-43650f767757") }, $db: "test" } planSummary: COLLSCAN numYields:5903 reslen:182 locks:{ Global: { acquireCount: { r: 11808 } }, Database: { acquireCount: { r: 5904 } }, Collection: { acquireCount: { r: 5904 } } } protocol:op_query 10019ms
2018-09-28T13:33:44.390+0200 I COMMAND  [conn8] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("e927734d-2de4-4d15-b958-5ed758860118") }, $db: "test" } planSummary: COLLSCAN numYields:5916 reslen:182 locks:{ Global: { acquireCount: { r: 11834 } }, Database: { acquireCount: { r: 5917 } }, Collection: { acquireCount: { r: 5917 } } } protocol:op_query 10017ms
 
2018-09-28T13:36:03.315+0200 I COMMAND  [conn11] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:48309 reslen:44 locks:{ Global: { acquireCount: { r: 96620 } }, Database: { acquireCount: { r: 48310 } }, Collection: { acquireCount: { r: 48310 } } } protocol:op_query 148950ms
2018-09-28T13:38:24.293+0200 I COMMAND  [conn10] command test.tweets command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47984 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 95970 } }, Database: { acquireCount: { r: 47985 } }, Collection: { acquireCount: { r: 47985 } } } protocol:op_query 140951ms

I tested this with Compass 1.16.0 beta 7.



 Comments   
Comment by Githook User [ 09/Apr/19 ]

Author:

{'email': 'durran@gmail.com', 'name': 'Durran Jordan', 'username': 'durran'}

Message: COMPASS-3157: Add maxTimeMS to query bar
Branch: master
https://github.com/10gen/compass/commit/56c00e1b583f74cbc7965a31d2dce1128f152716

Comment by Githook User [ 09/Apr/19 ]

Author:

{'email': 'durran@gmail.com', 'name': 'Durran Jordan', 'username': 'durran'}

Message: COMPASS-3157: Add maxTimeMS to query bar
Branch: COMPASS-3157
https://github.com/10gen/compass/commit/326274958457e3144233bee739ff44c9c1e18538

Comment by Githook User [ 09/Apr/19 ]

Author:

{'name': 'Durran Jordan', 'username': 'durran', 'email': 'durran@gmail.com'}

Message: COMPASS-3157: Add maxTimeMS to query bar
Branch: COMPASS-3157
https://github.com/10gen/compass/commit/739e21e36df0a8e526c618a6f3286f0b9ef1c4bb

Generated at Wed Feb 07 22:32:19 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.