[CSHARP-2794] Investigate slow redetection of xunit tests in IDE Created: 17/Oct/19  Updated: 31/Mar/22

Status: Backlog
Project: C# Driver
Component/s: Testing
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Vincent Kam (Inactive) Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Re-detecting the tests after compiling can sometimes take upwards of 3-4 minutes, which can be a drag when trying to iterate quickly. We should investigate to see if there's anything that can be done to improve this.



 Comments   
Comment by Robert Stam [ 23/Oct/19 ]

Another experiment I did was to unload (right click on project in Solution Explorer) some of the test projects.

That did in fact result in faster load times for Test Explorer because it is loading fewer tests.

Which projects are loaded/unloaded appears to be a local setting on your own machine and doesn't result in any files being changed and updated in github.

So duringĀ  a heavy debugging session that involves just one test project we can unload the rest and get faster testing turnaround times.

Comment by Robert Stam [ 23/Oct/19 ]

As another experiment I modified all the .csproj files to only target net452.

As expected that cut the number of tests approximately in half (because Test Explorer loads all the tests for each target framework) and cut the time to load the tests almost in half.

This is something to keep in mind when we add additional target frameworks in the future. Each new target framework will cause a substantial increase in the time Test Explorer will take to load the tests.

Comment by Robert Stam [ 23/Oct/19 ]

On the theory that the JSON driven test runners might be contributing to this slowness I compared how long it takes to load the tests with and without the JSON driven tests.

With JSON driven tests: 41858 tests loaded in approximately 1 minute

Without JSON driven tests: 37686 tests loaded in approximately 45 seconds

So while the JSON driven tests appear to be disproportionately heavy (they are 10% of the tests but they take 25% of the time to load) they alone do not account for the slowness. Rather it appears that the shear number of tests is the problem.

Notes: timing are approximate because Visual Studio does not report them, so I was using a stopwatch and timing how long after a full build finished the Test Explorer window finished loading all the tests. Test Explorer does not report it has finished, but one can deduce that it has finished when the progress bar stops moving.

Timings will of course vary depending on how fast the machine you run the tests on is. I also found a lot of variability of the timings on my own machine (+- 10% or so on each run).

Generated at Wed Feb 07 21:43:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.