provide an (extensibility) API for the Microsoft Test Manager
Currently we need to go quite some lengths in order to extend and customize the Microsoft Test Manager functionality (Planning, edition and running tests) because the functionality mtm(.exe) provides only overlaps to a certain extend to our usecases and right now it offers no public API which we can build upon.
We will not be able to provide extensibility within the Microsoft Test Manager client – it was not designed for this in the first place. Trying to go back and enable it now within the client is not something we plan to do.
That said, we are enabling extensions/contribution points within the Team Web access framework –
This should enable you to integrate/extend the TEST hub to cater to your testing needs.
Hence, I am marking this item for MTM as “Declined”.
Responses to some specific comments:
- Adding fields to test results: please vote for this using the following UV item:
Customizing Test Results work item type: https://visualstudio.uservoice.com/forums/121579-visual-studio/suggestions/5589345
- Exporting test suites/test cases – take a look at the “Export” capability added to the product – https://www.visualstudio.com/en-us/get-started/test/create-a-test-plan-vs (search for “export the test plan”)
I would like to use XML as data input instead of datarows
I would like to be able to add some fields in the test results tab as well as in work item type definition
I would like to be able to add some fields of the test results as well as work item type definition
I would like to be able to update fields in test results (like comments/notes) for particular TC using API.
Tushar Mhatre commented
I would like to be able to export test suites/test cases to word document as tables. The ability I need is to be able to generate a word document according to a given template. This ability is currently provided by HPQC. We need this ability to be able to migrate to MTM/Visual Studio as a generic solution.
Erik Saly commented
I would like the ability to export multiple test cases to Word, as tables.
Anna Russo commented
I would like if there was an extensions plugin manager like in Team Web Access
We have created a few tools using the API that we would, at bare minimum, like to make accessible from the MTM UI. The main reason is so that users don't have to keep track of numerous tool executables. Here are some of the tools: TestCaseExport, TestPointReport, TestPointMetrics. It would be even better if we could integrate the tool functionality into the existing MTM functionality(Ex. TestCaseExport - From the Test tab, click on TestSuite and export TestCase in the TestSuite w/option to export TestCases in sub-Suites as well. Thanks for considering.
Also addition extension points i'd like to see is a plugin type model. Being able to add new features via Plugins would be great, like the Test Scribe one created by MS.
@Jorg, sorry i didn't realise you replied to my comment. Although doable at the moment by syncing TFS with Jira it still require us to maintain a system we no longer use. It would be better if MTM acted as a stand alone system with greate TFS integration built in if you use it. I understand there there are many features in MTF that are TFS dependant and i would be happy to not be able to use these. The main thing would be Task CRUD sync between MTM and Jira(or some other system). If this was possible MTM would be a good stand alone Test tool. Features i think could be included are
1. CRUD of bugs / tasks in a separate task management system
2. Attaching of all the Data and Diagnostics collected (although video might want to be store in a different system like a file store)
This would go a long way. Additional nice to have would be
3. Automated Testing Execution
By doing this you open MTM to a whole new Market, MTM has some great features that no other system has, the action recording and VM integration for example. It sucks that we can't use it if we don't have TFS
Just wanted to reiterate this is a really good time to let us know what extensibility you are looking for.
I would like to add New tab To display Page to get few Testing related Statistic and Graphs from TFS
Jörg B. commented
the native defect management of TFS is tightly integrated in other parts of it aswell, particularly the Test Management component as well as the SCM part & interweaving/making this pluggable seems like a rather extensive and therefore probably unlikely change. However, as defects are work items, you can modify it to your needs (which might also mean strip the tfs-integrated down to a bare minimum and only link to a full-blown jira case from there (and vice versa maybe)). We've created something similiar for a very large, highly-regulated customer that does the same with their primary, validated non-tfs defect management system (Rational ClearQuest). Master in that case is also the external (ClearQuest) system, but shadow-copies are synced into tfs as tfs-native defects. In a next version we will extend this to two-way-sync (users can update defects on both ends), yet ClearQuest remains the master. And as I said, this is rather easily doable with TFS2010 already. The tricky part was only to integrate the business-specific ClearQuest customization into that picture. But beyond that, no problems regarding TFS.
I'd like to be able to change the Defect Manager to something like Jira
I would like to see extensibility support for the different Defect managers other than TFS such as Jira. Jira has far better project management abilities than TFS but MTM has some great features to. It would be good if we could select Jira as a plugin that would allow us to integrate bugs with jira bugs.
Jörg B. commented
Ben, I can only speak for our use cases, but our biggest pain-point with MTM is it's Test Runner tool not allowing us to hook into / customize the actual testing workflow in order to apply checks that our testing process defines.
For example once a test run contains at least one failed step, a bug must be created & attached to that test case. If that's not the case, the test run is void/inconclusive and either we spend manual time analyzing the test and writing a bug later on or in case the tester is no longer available, we re-plan the test case. We have several thousands of manual test cases per iteration to perform in a large variety of configurations (hundreds) and therefore we have to make sure actual test executions can be performed with as little as manual 'checks' after each test.
So basically the bare minimum, which would help us a lot, would be the possibility to have a cancellable 'On_Save' event or such being fired upon the User pressing the Save (&Close) button within the MTM Test Runner which we could handle and go on to iterate over the test steps, apply our business logic/checks and in case something has not been performed right, cancel the Save and inform the user what to do.
Ben Walters [MSFT] commented
Apolean, what kind of extensions would you like to build for MTM?