I suggest you ...

Test Explorer is severely limited

I have to assume the people who developed the VS2012/2013 test explorer view never actually had to use it in production. The test explorer view is horribly limited in comparison to the one in VS2010.

1. Running tests does not open the Test Explorer window until after all the tests completed running. There's literally no indication that anything is happening. VS2010 immediately opened the window to show you that the tests were running.

2. Where are the columns? Most importantly: the error message column. if I'm dealing with multiple test failures, I almost always just need to see the error messages at a glance. Having to select each individual test just to see the result is annoying.

3. Where'd the ability to run tests in context go? There's no way to run an individual test without going into the full test list, finding it, and then selecting it just to run it. This might be okay if you have a project with 10 tests, but a solution with thousands of tests makes this arduous. This needs to be fixed badly. Either bring back the "run tests in context" button or at least add some right click menus for "Run this test" and "Run all tests in class".

4. The progress bar is terrible. A five pixel high bar might be useful to someone with an 800x600 monitor, but for the rest of us a barely noticable progress bar is not enough to know that all the tests finished passing. ****, half the time the progress bar color doesn't even change. It just stays red/orange rather than changing to green.

Please fix this. It has made working with unit tests far more of a chore than it needs to be.

138 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Ross DickinsonRoss Dickinson shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    37 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #26: If you're running a lot of tests(hundreds, thousands) and see a lot of failures, there is no way to quickly see if they're all coming from the same test class or not. This was possible in the 2010 test viewer since it had adjustable columns with this info. You might see this and say "That's easy, just group results by class.", which leads to...

        #27: Group headers are useless. Text with the number of tests, that's it? It doesn't give a break down of passed/failed/inconclusive or give any indication at all about the status of the tests included. Now I suppose I could scroll through the huge list and wait till I find a plethora of red error icons, but that's not gonna happen due to...

        #28: No way to collapse/uncollapse all of the sub sections in the test explorer. I've got a solution with almost 5000 unit tests. The "Group by Class" breakdown gives me a tree with hundreds of collapsed nodes. I know I'm not gonna click through every single one of them just to open them.

        #29: #28 wouldn't be a problem if there was a way to group by one thing and then sort by another. Like group by class, then sort by outcome so all classes with failed tests show up at the top.

        I feel like, after updating this post for over a year, that it should be fairly obvious that using a Tree View to view test results is very cumbersome and limited. Test results are one of those areas where a typical grid makes the most sense for parsing information quickly.

      • Jaya RayasamJaya Rayasam commented  ·   ·  Flag as inappropriate

        I would love to have something like output window while running the test to know on what step of the test it is currently running. Can we get that feature?

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #25: If you want to run all tests except for one, you can't just do ctrl+a and then unselect the offending test. You also have to unselect whatever parent tree node it's in because otherwise the test will be ran anyway. This would not be an issue if we had sortable columns instead of a treeview control.

      • Tudor TurcuTudor Turcu commented  ·   ·  Flag as inappropriate

        The progress bar is again visible in the Update 4 for VS 2013 (they made it much thicker again so no it's possible to see the progress without a loupe)

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        Going back to #11 again: We have a backup script at work that deletes that TestResults folder before backing up our source code. So this ends up deleting the mdf file that VS makes in there. Why it doesn't just recreate the file(like it does after restarting VS) is beyond me.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        Going back to #23 again: Here's a random sampling I get from hitting "Run All" in the Test Explorer window. For the current solution I have open, there are 4474 tests.

        ------ Discover test started ------
        ========== Discover test finished: 3838 found (0:00:03.811) ==========
        ------ Run test started ------
        ========== Run test finished: 2645 run (0:01:27.939) ==========

        So it found 3838 files, not finding about 500 tests, and then didn't run another 1000+ of them.

        If I rebuild the solution, it will then say it found 4474 tests, even though they were there in the first place. I'm assuming the discover tests count is only for the projects in the solution that were just built, but it doesn't explain why chunks of tests get ignored when attempting to do a Run All.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        Going back to #23: At some point, "Run All" will only run all the unit tests for a single test project. It completely ignores all the other test projects in the solution.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #24: The Deploy directories in the TestResults directory do not get deleted on failed tests. That would be fine, except it's ignoring the "Limit number of old test results to" setting. I have it set to 1, I have never seen a need for keeping old test results. This is just wasting drive space for me on huge projects.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        Carl: To correct myself: "It just stays red/orange rather than changing to green when all the tests have passed after a previous failed run". Having said that, I think it was fixed in one of the updates.

      • CarlCarl commented  ·   ·  Flag as inappropriate

        "It just stays red/orange rather than changing to green."

        This means that your tests have failed...

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #23: If you hit the "Run All" button and then open the the test explorer while it's still running, the progress bar is grayed out and makes no indication of progress. This only seems to happen if it's the first time opening the Test Explorer after opening Visual Studio.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #22: Occasionally "Run All" does not actually run all the tests. It doesn't give any indication that it didn't unless you decide to scroll through all the tests to see if any of them are grayed out. The only fix is to reopen the solution. Rebuilding doesn't fix it, even though the test window will say it has found all of them.

      • JayJay commented  ·   ·  Flag as inappropriate

        #21: The biggest problem for me is that when I have multiple test properties with overlapping values, the Filter can't understand I want to map Trait/Property A with Value 1. It brings up all tests where either the key or the value match A or 1. I have thousands of tests with 20 different Traits, and this UI not only doesn't scale, it's basically broken.

        http://stackoverflow.com/questions/24807611/distinguish-unit-test-traits-or-test-properties-with-overlapping-values

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        Going back to #11, this seems to happen to me when I leave Visual Studio open overnight. Without fail, the moment I try to run tests for the first time I get hit with "Underlying provider could not be opened" errors.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #18: When grouped by class and all of the class sections are collapsed, opening any section and then removing all of the tests in that section causes the top-most section to suddenly expand.

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #16: Can't remove all tests from a playlist by selecting their grouped section. ex: I can't group tests by project and then right click the project heading and remove the tests. I have to highlight all of them first and then right click. That's a pain, especially if you want to remove multiple grouped areas(see #15 for why I even have to do this in the first place).

      • Ross DickinsonRoss Dickinson commented  ·   ·  Flag as inappropriate

        #15: There isn't a way to group the tests in the order that they ran. I'm currently stuck trying to figure out why 6 tests suddenly fail on me when they're run together with all the other unit tests in the project. I can't even begin to try and narrow down which tests ran before the failing tests so I can find out the cause of it.

      ← Previous 1

      Feedback and Knowledge Base