Uploaded image for project: 'Compass '
  1. Compass
  2. COMPASS-3157

Enforce maxTimeMS for every query sent to the server

    XMLWordPrintable

Details

    • Story
    • Status: Closed
    • Major - P3
    • Resolution: Done
    • 1.16.0-beta.7, 1.15.4
    • 1.19.0
    • CRUD, Query, Schema
    • None
    • 3
    • Needed
    • The querybar now contains a new box to specify maxTimeMS (see attachment)
    • Iteration Antelope, Iteration Bison

    Description

      Currently query filters that can be entered in the "Documents" panel don't use the maxtimems parameter. It is possible with non selective filters not supported by indexes to cause to run very heavy queries (COLLSCAN) on the MongoDB server causing possibly performance issues for the other clients. The parameter maxTimeMS is not set in the count and in the find commands sent.

      For example I loaded 5.7M tweets on a test MongoDB instances and I tried to use the filter

      {source: "a"}

      which causes a full collection scan over 5.7M documents:

      2018-09-28T10:42:42.588+0200 I NETWORK  [conn6] received client metadata from 127.0.0.1:51337 conn6: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.6.6" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "17.7.0" } }
      2018-09-28T10:44:52.172+0200 I COMMAND  [conn5] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:47574 reslen:44 locks:{ Global: { acquireCount: { r: 95150 } }, Database: { acquireCount: { r: 47575 } }, Collection: { acquireCount: { r: 47575 } } } protocol:op_query 137667ms
      2018-09-28T10:46:53.077+0200 I COMMAND  [conn1] command test.tweets appName: "MongoDB Compass Beta" command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("884572c0-9842-4d2f-ada2-72250fc64b9a") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47243 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 94488 } }, Database: { acquireCount: { r: 47244 } }, Collection: { acquireCount: { r: 47244 } } } protocol:op_query 120877ms
      

      The query is limited to 20 documents but has no maxTimeMS set.

      If you use the schema tab and you run the same filter, maxTimeMS is used, but I noticed that the same queries are then run without the timeout set:

      2018-09-28T13:33:44.380+0200 I COMMAND  [conn10] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("bbda3184-e246-44bd-a5dd-43650f767757") }, $db: "test" } planSummary: COLLSCAN numYields:5903 reslen:182 locks:{ Global: { acquireCount: { r: 11808 } }, Database: { acquireCount: { r: 5904 } }, Collection: { acquireCount: { r: 5904 } } } protocol:op_query 10019ms
      2018-09-28T13:33:44.390+0200 I COMMAND  [conn8] command test.tweets command: count { count: "tweets", query: { source: "a" }, maxTimeMS: 10000, lsid: { id: UUID("e927734d-2de4-4d15-b958-5ed758860118") }, $db: "test" } planSummary: COLLSCAN numYields:5916 reslen:182 locks:{ Global: { acquireCount: { r: 11834 } }, Database: { acquireCount: { r: 5917 } }, Collection: { acquireCount: { r: 5917 } } } protocol:op_query 10017ms
       
      2018-09-28T13:36:03.315+0200 I COMMAND  [conn11] command test.tweets command: count { count: "tweets", query: { source: "a" }, skip: 0, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 numYields:48309 reslen:44 locks:{ Global: { acquireCount: { r: 96620 } }, Database: { acquireCount: { r: 48310 } }, Collection: { acquireCount: { r: 48310 } } } protocol:op_query 148950ms
      2018-09-28T13:38:24.293+0200 I COMMAND  [conn10] command test.tweets command: find { find: "tweets", filter: { source: "a" }, limit: 20, returnKey: false, showRecordId: false, lsid: { id: UUID("f153a56e-d826-4256-bc41-3402c718bf42") }, $db: "test" } planSummary: COLLSCAN keysExamined:0 docsExamined:5712169 cursorExhausted:1 numYields:47984 nreturned:0 reslen:99 locks:{ Global: { acquireCount: { r: 95970 } }, Database: { acquireCount: { r: 47985 } }, Collection: { acquireCount: { r: 47985 } } } protocol:op_query 140951ms
      

      I tested this with Compass 1.16.0 beta 7.

      Attachments

        Issue Links

          Activity

            People

              durran.jordan@mongodb.com Durran Jordan
              emilio.scalise@mongodb.com Emilio Scalise
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: