Allow tests to be grouped by class, category, and namespace in Test Explorer
The Visual Studio team has obviously taken the time to add grouping to the new Test Explorer since we can group by test outcome and duration. However, sometimes you need to run a group of tests such as all tests in a category or namespace and this is not currently possible with Test Explorer. It would be nice if the grouping options were expanded to include at least class, category, and namespace. Then it would be possible to run all tests in a class, category, or namespace from the Test Explorer.
Hughes Hilton
shared this idea
Visual Studio 2012 Update 1 http://www.microsoft.com/visualstudio/eng/visual-studio-update has added enhancements to support both grouping and filtering by Project and Traits (category). Use of these features is detailed in this blog post, http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using-traits-in-the-unit-test-explorer.aspx
Group and filter by Class is completed and will be available in Update 2.
26 comments
-
Anonymous
commented
Please add some sort of "immediate" display so that I can see what test is running?
-
JamesT
commented
I can't believe this wasn't part of the update, this is such an enormous nuisance. I wish I'd never started unit testing in VS2010...
Is there any way MS could just reactivate the original test list editor so we can use it again? Or as a plugin? It seems this is just not a priority. it also seems astonishing we're still waiting, since the feature seems just so dead simple...
-
PaoloPablito
commented
Microsoft promised improvements to the Unit Test interface for the Update 1, in fact now the Test Explorer has new filtering capabilities. I still cannot filter by namespace like in good old Test List Editor, but I can simulate it by filtering on Full Qualified Name.
But other things in the Unit Test world of VS 2012 leave me disappointed, they are the same things that left me disappointed before applying Update 1:
- it looks slower to me. I created two copies of the same solution and opened it with VS 2008 and 2012 on the same Win2008R2sp1 machine (a virtual machine on VMWare LabManager, not a real machine, but it's the SAME machine, so they are on an even footing). I selected a bunch of random tests and executed them on both VS 2008 and 2012. VS 2008 executed my tests in a few seconds, with VS 2012 I had to wait at least 18-20 seconds to see the results. I retries the same tests multiple times but I got the same results. It looks like the problem is the "initial sprint", not the actual test execution. Even if I select a SINGLE test, it takes at least 5 seconds to show me the results, and then Test Explorer says that the actual execution took only 129 milliseconds! Is there some VSHOST trick to activate?
- the beloved Test List Editor is STILL deprecated and cannot run selected tests
- the Test Explorer still doesn't allow you to select individual tests with checkboxes like the good old Test List Editor. I often find clicking checkboxes more convenient than CTRL+clicking the desired tests.
- there's still no test toolbar (or at least I cannot see it in the toolbar list)
- When I'm in the source code and I run a test, I'd like to be brought immediately to the window that shows my test running, it does not happen in VS 2012 (but it used to happen in previous versions). It happens later when the test finishes: but if there's a filter in Test Explorer, I cannot see the results, I have to clear the filter.
- If I click Debug or press F5 in previous versions, all tests in the current PROJECT (not solution) are executed. VS 2012 does not execute anything, it complains that this is a DLL and cannot be run.
- the Test Result windows does not work as it worked in VS 2008 and 2010, it does not show anything. Test results are shown in the Test Explorer exactly like before applying the Update 1, this view is clumsly and I don't find it useful like the good old Test Result window. To see the error messages, I have to click on single failed tests, I'd like to have a look at error messages at a glance to get the big picture. This is very important for me when I try to use a TDD-like approach.
Is there a way to customize the data shown by Test Explorer? Where? (VS 2010 has a "Test Tools" branch in the Option dialog, but VS 2012 hasn't it).
Sorry for the long list of complaints (which, by the way, I already posted on Brian Harry's blog) but I have to say what I think. I still prefer the testing interface of VS 2008 and 2010 (and I prefer 2008 when it does not force me to click Refresh in the Test List Editor)
-
Joshka
commented
Note: Oleg Sych's answer at http://stackoverflow.com/a/13588093, mentions that the group by class did not make VS 2012 Udate 1. This is really really important as the class is the most useful grouping of tests for many of out projects.
-
Mikhail Poliakov
commented
In VS 2010, I often ran all tests in a test class (Ctrl-R,C), now it seems not working.
Another pain is that the TestContext's output is unusable now, since the output content is not line -wrapped, you can't scroll it, copy/paste the result or do anything at all with this.
-
Terje Sandstrøm commented
This is now in the CTP for Update 1 http://www.microsoft.com/en-ie/download/details.aspx?id=34818 , and Update 1 should be out real soon now. Blogged it here : http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/09/how-to-manage-unit-tests-in-visual-studio-2012-update-1-part-1-using-traits-in-the-unit-test-explorer.aspx
-
dmx707
commented
I've never used ReSharper test window in VS 2010, I just didn't need it. VS 2010 Test Explorer had every thing I need. But now I use this plugin every day, because VS 2012 Test Explorer has literally NOTHING.
-
Willem
commented
Same problem here... We have differrent TestProjects for Unittest and Intergration/database tests. In VS2010 it is possible to run tests per project. In VS2012 it is not possible.
-
Howard Richards
commented
I used VS2012 for development, but load VS2010 for unit testing.
That kind of says it all - the testing functionality in VS2012 is primitive.
-
Thomas K
commented
Just to make the pain that the changed UI has brought us clear:
We have a big solution with > 13000 tests. Some of the tests fail, but we still need to run > 6000 of them. We tried with test lists, but maintaining them is not practical. So we switched to TestPriority(X) and use /minpriority:20 to filter the known failures away. Worked with VS2010.
There is simply no help from the UI at this point in time and running tests is no fun. Also, because the test results UI has changed to the point where this approach is not feasible. And no, we do not want to mark 6000+ tests with another attribute in order to include them; for us, it is more practical to mark only those tests that we do not want to run.
The current UIs with their emphasis on tree views might be cool for smaller numbers of tests/files, but is impractical with large solutions.
VS2012: Two steps ahead, but one backwards and shoes lost! -
Olexandr Kravets
commented
I am very disappointed too.
Without previously available features, Visual Studio Team MADE huge step back.
UI, icons, and test explorer ...Also before I was able to see, which test is currently running. Now - NO ...
-
Tommi Kaskinen
commented
I can't believe what MS has done with 2012. Compared to VS 2010 test view, test explorer
is unusable. I have multiple projects in my solution and i always sort them at least by project.
I've always used test view for running and finding correct scripts.We cannot use VS 2012 until this issue is fixed. MS development has taken a major leap backwards.
-
Pros
commented
Allow for partial copy/paste from a unit test run details -- ie. I want to copy a path or exception name from a unit test error, and not the entire test run details.
-
Joshua Weber - msft
commented
Thank you everyone for the feedback on this feature. We agree that different grouping options are essential for managing tests. Realize that the new Unit Test Explorer has been written to be test framework agnostic. To fully support 3rd party testing frameworks required a significant change to the underlying test execution architecture, hence the removal of the test list editor. It also has resulted in careful implementation of grouping options.
In the upcoming Visual Studio update we have added support to allow for grouping by project and trait. Trait being the generalized framework agnostic grouping, supporting framework specific method decorations (testcategory being one in the MSTest framework). We are also looking into further supporting class as a grouping option.
Joshua Weber
VS ALM Tools - PM -
James Treworgy
commented
How about just un-deprecating Test List Editor until this is resolved? It worked. It wasn't the prettiest thing in the world, but hey, you could actually find the tests you were looking for.
I'm not sure if this was just a massive oversight, or there's some technical reason why Test List Editor could no longer be supported and someone just didn't finish Test Explorer in time. But it really stinks when something you use about about as often as you hit the "enter" key is suddenly gone. Don't give us new toys then take them away!
-
Charles Reid
commented
This is absolutely a must have. Please bring this back as soon as possible as I am basically forced to go back to 2010 until this is fixed.
-
Doug
commented
Every change to the test interface has been a step to backwards (or out of the airlock).
I would prefer not to run all of the slow tests (+1 min each) when making a change to unrelated code. I would prefer to be informed of test progress pass / fail / not run yet as the testing progresses so if I cancel I got useful information.
Is this designed for "Hello World"?
-
Anonymous
commented
Whats the use of having test category when we cannot group it by category to run unit tests, test category can also be removed.
-
Gareth Young
commented
One of the downsides of the new Test Explorer interface is that if running a large number of tests it is much ****** to see which test is in progress - in VS2010 the test in progress would jump up in the test results window and IIRC change the symbol to something representing 'in progress'.
Please alter the interface so that it is obvious which test is currently running - we tend to use the unit test system to do functional testing too and sometimes these tests can hang. In that situation it is useful to know which test caused the hang so that it can be disabled in the next run and have a TFS bug entered. At present the test run may hang and you have to cancel the entire run and even then may not be any the wiser which test caused the failure.
-
Dion Siswadi commented
would absolutely support this missing feature.
