Team Test - Improve the usability of Microsoft Test Manager
There are many usability issues with Microsoft Test Manager, most of the centered around the test editing and running expieriences. One major gripe is the inflexiblility of the test editing user interface: can't resize the panes, can't enter multi-line text (I actually created a separate tool that originated due to this limitation: http://teststepseditor.codeplex.com).
The test runner can lose attachments forever if you lose a network connection (although this has improved a bit with recent patches).
(By the way, the most appropriate category seemed to be "Visual Studio Lab Management." Test Management is not listed, but "Test & Lab Management" are grouped together in the site description bullets up top).
We have done a lot of work w.r.t. test steps editing, bulk marking of tests, performance, etc. The experience should definitely be much improved in the next release. Marking this as completed. Please open new threads for specific asks.
complete web testing with sample source code
with writing test cases and test executions.
I am using Test manager 2010 and want to execute test cases existing in a project in bulk rather than running them individually which is a tedious and a time consuming task. Is there any way to run all the test cases together and then export them in Excel having the results like Pass/Fail with it?
Looking forward to your kind response. Thanks
Bruce Cutler commented
Good - I will check out Dev 11 before making any more comments. in particular I would like to drag and drop test cases from one static suite to another static suite. I would also like to multi-select test cases/suites in order to delete or move them.
The above link talks about some of the imrovements in MTM in VS11. Multiline test steps, rich text formatting of test steps, resizing of area between steps and parameters and many other improvements have been made.
-Nivedita Bawa, Program Manager, Microsoft Visual Studio
"change the Area/Iteration in bulk for a test suite or a number of items in a test suite"
I know this is an old thread, but since I was looking for this information also I thought this might be of use to the next reader.
It works quite well to query for the test cases in Visual Studio, Make sure the columns you want to edit are included, then "Open Query in Microsoft Excel." In Excel, you can select the appropriate Area from a list box individually, or cut and paste it to the entire list. When you are done simply click "Publish" and waa laa! All the areas are updated.
lior regev commented
You are absolutely right!
I encourage you to add your comment here: http://visualstudio.uservoice.com/forums/121579-visual-studio/category/30927-visual-studio-test-and-lab-management
and let others vote for making the change in the future.
Ian Watt commented
Why is no-one bothered about the inability to extract and report on Test Suites and the Test Cases within them. Test suites are an excellent way of structuring and grouping test cases within Test Plans particularly in hierarchies. This gives a Test Case context within a plan. There does not appear to be any way to report on the Test Cases within this grouping/hierarchy. Test Scribe simply lists all test cases (and steps) within a choosen suite, Test Extractor again lists all test cases. Both do not identify the suite they belong to. TFS and Test Manager querying does not appear to offer the suite as either a filter or display field. Why has this important piece of data which helps organise Test Cases been ignored??
Anna Russo commented
Visual Studio 2010 SP1 TFS Compatibility GDR for Microsoft Test Manager improvements:
1) Most testers have been accustomed to having multi-line support in Excel for managing their test cases. So it's exciting that now Microsoft Test Manager supports multi-line test steps with ALT+ENTER(just like in Excel!).
2) If you are in Microsoft Test Manager and have found that the Test Steps view is limiting, that has now changed. Once you have Visual Studio 2010 SP1 and the GDR installed you will see that the number of Test Steps that you view at once has increased!
Here are the instructions to install the GDR http://improvingsoftwarequality.blogspot.com/2011/10/multi-line-support-for-test-steps-in.html
David Ricciardelli commented
I have many issues with the usability of this product. My biggest gripe is manually running a test case and not being able to change the pane size or minimize it. I would appreciate it if Test Manager would allow me to use the screen resource the way I want to use it.
Please add the ability to find test cases in a test plan and handle duplicates. I have a test plan with about 7000 test cases (all automated) and a handfull of duplicate test cases.
To find the duplicates I need to look in each test suite to see if the duplicate test case exists.
Bugge T Jensen commented
Any planned relase date. I realy look forward to any improvement on the edit :)
Allen Feinberg commented
You've started work on this...but especially around test case editing? Surely you'll focus on the boarder issue. Using MTM is significantly painful. Try managing dozens of test configurations. Try managing dozens of builds and dozens of build types. Try managing the test hiearchies...the list is endless. Try managing the versioning of test cases. Try managing the auditing of Test Plans. Try editing with coded ui tests that aren't recorded using MTM. I hope you're are serious, truly SERIOUS, about fixing the USABLITY of MTM and now just the FUNCTIONALITY of MTM. Working with v1 for anything longer than an hour is like trying to eat shards of glass.
Josh Wright commented
That may help. However I still wouldn't have the slick UI of TFS WA for just updating one field across 20 or 30 work items. I'd have to Open Excel, Find the field i wanted to change if it were hidden, change the value in one field, then copy that to the other 20 or 30 items, then publish back. TFS WA, I select items, click Bulk Edit, Pick field, Pick Value, Click OK.
You could just let me see my test suites in TFS WA and I'd be incredibly happy!
Navigating around the app has always shown problems. As woody suggested it takes some getting used to. If you spend all day using it i have seen cases where you unintentionally had about 40 different items. I often confuse the run section with the plan section and will try to run or open an item when your in the wrong section. Also i manage the labs between projects and to change projects there is a very long process including selecting an iteration that has nothing to do with what you are doing at the time. Another example i can think of is you go to plan, select properties to check to overall status, go to test and run some test, the next time you go to plan you don't see the usual plan screen but the properties instead, this often startles you as you try to figure out what the heck is going on.
The layout takes some getting used to and the multiple windows concept (where you can have like 9 windows open) is irritating. By layout, I mean the way you get to test results, test runs. The difference between opening a test case and analyzing its results. It's not intuitive and I have heard many users having a hard time finding things in the UI.
Anu MSFT commented
Josh - if I were to build Excel integration like the query editor in VS, where a button would open these test cases in Excel and let you bulk edit and publish back, would that solve your problem?
Anu MSFT commented
Usability is a broad area - I'd love to understand more. Are these 2 your top concerns?
1. Multi line text in test steps, more flexibility in the test case editing pane
2. Work better in transient network conditions
We have addressed a part of 2 in recent times like you note. We are fixing 1 for the next release by supporting multi line text and doing a bunch of fixes in the test step grid.
Josh Wright commented
Also allow for the 'Bulk Edit Selected Work Items' that is available in TFS WA. Specifically I'd like to be able to change the Area/Iteration in bulk for a test suite or a number of items in a test suite.