[COMPASS-6332] Compass export feels really slow Created: 25/Nov/22  Updated: 29/Oct/23  Resolved: 25/Nov/22

Status: Closed
Project: Compass
Component/s: Import/Export
Affects Version/s: None
Fix Version/s: 1.35.0

Type: Bug Priority: Major - P3
Reporter: Le Roux Bodenstein Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Story Points: 2
Documentation Changes: Not Needed
Sprint: Iteration Porpoise

 Description   

While using import and export locally I realised that export is many times slower than import which made no sense to me.

I noticed two things:

1) The progress updates are not happening at 250ms intervals as we specify here, but in fact seem to happen in a tight loop (probably every row?):
https://github.com/mongodb-js/compass/blob/92270b35885e90f6eb7b153275848f7a2be21278/packages/compass-import-export/src/modules/export.ts#L501-L511

See https://mongodb.slack.com/archives/G2L10JAV7/p1669366262289589

2) At least locally in development the dev tools console breaks during export of large collections. You have to close it and reopen it to get it to work again. I logged every time we update the progress and only the first few logs would show up even though the progress keeps updating.

See https://mongodb.slack.com/archives/G2L10JAV7/p1669369902773209



 Comments   
Comment by Githook User [ 01/Dec/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 30/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 28/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 28/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 28/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 28/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 25/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Comment by Githook User [ 25/Nov/22 ]

Author:

{'name': 'Le Roux Bodenstein', 'email': 'lerouxb@gmail.com', 'username': 'lerouxb'}

Message: fix(export): Speed up export COMPASS-6332 (#3824)

  • Speed up export
Generated at Wed Feb 07 22:42:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.