[COMPASS-3269] Poor performance when using Views in Compass Created: 19/Nov/18  Updated: 29/Oct/23  Resolved: 21/Dec/18

Status: Closed
Project: Compass
Component/s: Compass
Affects Version/s: None
Fix Version/s: 1.18.0

Type: Bug Priority: Major - P3
Reporter: Ivan Grigolon Assignee: Durran Jordan
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: JPEG File COMPASS.jpeg    
Issue Links:
Related
related to COMPASS-3347 Product Design: Empty State for Last ... Closed
Sprint: Iteration Unagi

 Description   

A user is reporting a timeout problem in Compass when clicking on a View where the underlying collection has around 15.2m documents and a total size of 517.2GB. Document's average size is 35.6KB.

I have been able to reproduce the same problem in my testing environment.

The problem only appears when the original collection is fairly large, (in my testing I could only generate the problem when I had 5M documents in it) and the View is not using a $match stage to filter out the content.

Compass produce a timeout visible in the UI (see attachment)

connection 0 to localhost:27018 timed out

When the view is clicked in the UI, an aggregation pipeline is used to count the number of documents on the underlying collections. When the time to run this count command goes above a threshold that appears to be around 6 minutes, Compass timeout the operation.

The following count operation is visible in the log:

2018-11-20T08:28:35.583+1100 D COMMAND  [conn74] run command test.$cmd { count: "v_rides", query: {} }
2018-11-20T08:28:35.583+1100 D COMMAND  [conn70] run command test.$cmd { count: "v_rides", query: {}, skip: 0 }
2018-11-20T08:35:16.744+1100 I COMMAND  [conn70] command test.v_rides appName: "MongoDB Compass" command: count { count: "v_rides", query: {}, skip: 0 } planSummary: COLLSCAN keysExamined:0 docsExamined:5034881 cursorExhausted:1 numYields:39795 nreturned:1 reslen:40 locks:{ Global: { acquireCount: { r: 80268 } }, Database: { acquireCount: { r: 40134 } }, Collection: { acquireCount: { r: 40133 } } } protocol:op_query 401149ms
2018-11-20T08:35:16.754+1100 I COMMAND  [conn74] command test.v_rides command: count { count: "v_rides", query: {} } planSummary: COLLSCAN keysExamined:0 docsExamined:5034881 cursorExhausted:1 numYields:39778 nreturned:1 reslen:40 locks:{ Global: { acquireCount: { r: 80234 } }, Database: { acquireCount: { r: 40117 } }, Collection: { acquireCount: { r: 40116 } } } protocol:op_query 401160ms

I believe this is an undesirable behaviour because:
1. If I click on the underlying collection documents are showed immediatelly
2. If the timeout is caused by the count operation taking too long, this could be improved.
3. Compass should be able to detect that a view is used and adapt its logic



 Comments   
Comment by Githook User [ 21/Dec/18 ]

Author:

{'username': 'durran', 'email': 'durran@gmail.com', 'name': 'Durran Jordan'}

Message: COMPASS-3269: Fix pagination for readonly views
Branch: master
https://github.com/10gen/compass/commit/30e280196f79423bb1be9be087b228101763bcc8

Comment by Githook User [ 21/Dec/18 ]

Author:

{'username': 'durran', 'email': 'durran@gmail.com', 'name': 'Durran Jordan'}

Message: COMPASS-3269: Fix pagination for readonly views
Branch: COMPASS-3269
https://github.com/10gen/compass/commit/39ce9f36cc6548b5cc0368a6f276e60e703e357a

Comment by Githook User [ 13/Dec/18 ]

Author:

{'username': 'durran', 'email': 'durran@gmail.com', 'name': 'Durran Jordan'}

Message: COMPASS-3269: Add maxTimeMS where count is used
Branch: master
https://github.com/10gen/compass/commit/c7f1d07ec6a0997fd48e371afbd57fbd32ba33a5

Comment by Githook User [ 13/Dec/18 ]

Author:

{'username': 'durran', 'email': 'durran@gmail.com', 'name': 'Durran Jordan'}

Message: COMPASS-3269: Add maxTimeMS where count is used
Branch: COMPASS-3269
https://github.com/10gen/compass/commit/3b039d6cb7a397ccec692443136c7a399df6247f

Generated at Wed Feb 07 22:32:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.