Test Explorer is severely limited
I have to assume the people who developed the VS2012/2013 test explorer view never actually had to use it in production. The test explorer view is horribly limited in comparison to the one in VS2010.
1. Running tests does not open the Test Explorer window until after all the tests completed running. There's literally no indication that anything is happening. VS2010 immediately opened the window to show you that the tests were running.
2. Where are the columns? Most importantly: the error message column. if I'm dealing with multiple test failures, I almost always just need to see the error messages at a glance. Having to select each individual test just to see the result is annoying.
3. Where'd the ability to run tests in context go? There's no way to run an individual test without going into the full test list, finding it, and then selecting it just to run it. This might be okay if you have a project with 10 tests, but a solution with thousands of tests makes this arduous. This needs to be fixed badly. Either bring back the "run tests in context" button or at least add some right click menus for "Run this test" and "Run all tests in class".
4. The progress bar is terrible. A five pixel high bar might be useful to someone with an 800x600 monitor, but for the rest of us a barely noticable progress bar is not enough to know that all the tests finished passing. ****, half the time the progress bar color doesn't even change. It just stays red/orange rather than changing to green.
Please fix this. It has made working with unit tests far more of a chore than it needs to be.
Manfred Lange commented
Another short coming: Assume you want to debug both, your ASP.NET application and also the test. This is a scenario that you may run into for example if you want to diagnose a problem with a Selenium WebDriver based test. As you step through the test you want to see which step is the one that causes the system under test to behave incorrectly.
The test explorer will gray out all commands for tests if any project in the solution is already running under debugger. A possible work around could be to launch the same solution twice, i.e. have two instances of Visual Studio running. However, other tools allow debugging tests even if one (or more) of the projects in the solution are already being debugged.
This is a shortcoming as well. I can't see a technical reason why this shouldn't be possible. I can name at least one commercial tool that supports this scenario.
Manfred Lange commented
VS 2017 15.2 (so fairly recent): Grouping by namespace lists all tests in that namespace but does not include class names. In one case we have dozens of classes in a namespace and across them almost 600 tests. Alternatively we can group by project but that leaves out the class names, too, so it doesn't help either. Next option is grouping by class name. We have different categories of tests and the name of the class may be duplicated. The namespace is not included when grouping by class name. Again, you are left to "guess" which one it might be.
Is this tools seriously being used by anyone for more than a "toy" project? Or is this an area that Microsoft stays away from as to not interfere with commercial offerings from partners? If so, a clear statement by Microsoft would be nice, so that everybody can plan accordingly.
Yes test explorer in VS2015 Pro is not good.
It's VS 2017 now, 4 years have passed, and Test Explorer is still barely usable.
1. Logs are still truncated (why on earth are they truncated? why do we need to copy it to notepad?!)
2. There is no way to see a unified log of all tests that were run, which might be useful. E.g., logging from the [AssemblyInitialize] goes into the first test, and it's not easy to tell, which test ran first.
3. Logs are not using monospaced fonts.
4. Test Explorer is rather slow, even on beefy machines and small projects with less than ten tests.
Pratap Lakshman [MSFT] commented
Thank you for your continued feedback. Please see my comment below from Aug 2016. I will take another shot a compiling the feedback that we did not address then, and get back to you all with a proposal on how we will address them (as we did the last time).
The UV item has a lot of suggestions/asks in it and as such it might well be that we may not address "absolutely all" of them. Consequently it might not be possible to mark this "completed" ever. Going forward, I request that you open individual UV items for your ideas. That granularity will not only allow others to upvote specific ideas, but also make it easier for the ideas themselves to be tracked and updated.
Visual Studio Team
Gustavo Russo commented
After testing VS 2017 RC, I feel the Test Explorer still way behind the one in VS 2010.
I specially miss a simple way to reduce the current list of tests to a more concrete list to work with (It would be like creating a new playlist but without saving it)
Performance is still a problem that slows down the entire TDD cycle.
Kevin Cherry commented
One of my main concerns is that of hierarchical test categories. In VS 2010, I had a project with around 700 unit tests and they were split up into categories. Those categories where then split into sub-categories, which themselves had sub-sub-categories, etc. With 700 tests, you had to do something like this to stay organized. I could then run either an entire root category, or a sub-category, or a sub-sub-category etc. Now in VS 2015, all we get is root categories??? Do you honestly feel that 700 unit tests can be sufficiently organized by a single level of categorization? I agree whole-heartedly with the original post and with him saying that the VS team has never actually had to use this testing functionality on an actual project. Hey Visual Studio team, why don't you try using your own tools for an actual, real, complex task and see if the features are adequate enough and see if the workflow is smooth and efficient enough. Things look great on paper don't they? Until you actually have to use this mess of a unit test suite.
Pratap Lakshman [MSFT] commented
This item has a lot of suggestions/entries.
Just want to let you know that we have made a start as called out here: bhttps://blogs.msdn.microsoft.com/visualstudioalm/2016/08/05/evolving-the-visual-studio-test-platform-part-2/
Visual Studio Team
After considering porting big solution with thousands of test cases from 2010 -> 2015 we decided not to do that because of our devs complained about bad test management in 2015. Thank you MS for saving us some costs.
Adam Meltzer commented
related feedback (which was closed): https://visualstudio.uservoice.com/forums/121579-visual-studio-2015/suggestions/3473855-bring-back-the-vs-2010-test-explorer-and-test-resu
Right click - run this test was great. Why did it have to go?
Run All randomly stops and does not actually run everything it has discovered earlier.
'Configure continuous integration' message is annoying and cannot be removed to save screen real estate.
Don't know if it has been mentioned but its not possible to copy/paste something directly from the popup of the error icon in VS2015, if you have a exception for example.
You have to go to the testexplorer, maybe change the filtering to Outcome and the select copy of the desired test. Very uneconomic and time expensive.
#40 - Error/Exception are not shown properly
#41 - Report for Test with Iteration is messed up
Did not like at all this Test Explorer.
Run tests in context is: Ctrl+R, T
Debug tests in context: Ctrl+R, Ctrl+T
If you want a button just add a button with the customise option.
No way to clear test results either. Need to clean and rebuild the entire solution and then way for tests being re-discovered. Losing time. Simple 'clear all results' would do. And yes, where did right click - run test in CODE WINDOW go?
Timothy Klenke commented
+2 for #36. When grouping tests by Class the tree should include the namespace hierarchy as tree nodes, class name as the last branch, and test method names as the leaves. This would allow to run all tests in any namespace.
I use the log all the time for GUI testing, why take the feature away! Bring it back, Please
#40: Test Explorer to remain pinned to the main screen on restart (VS2015 professional)
Exactly, exactly, exactly! I'm so glad to see I'm not that only one that can't stand the text explorer changes after 2010 - don't "fix" something that isn't broken! Seriously VS 2012 made me want to either quit coding or go back to open source development and take a massive pay cut.
Please show in-progress for tests that are running. Can't you move back to the 2010 unit test view and features.