Allow tests to be paused, allow other tests to be run, and then resume the paused test case
MTM provides the ability to pause a test case, but I cannot run any other test case (or else the paused test case data is lost). I can only resume the paused test or discard it to create a new one.
Testers would like to execute on a particular test case for a while, if they face a blocker, they should pause this test case and pick up others for testing. Later, they should be allowed to resume the paused test case
This is necessary because if you need to stop manual testing for some reason you can continue where you left off without having to run the whole test case again and do unnecessary rework.
Let's say I come across an issue halfway across my testing that requires an answer from management. Ideally I could pause my execution of the particular test case, send an email requesting clarification and continue testing other test cases while I wait for an answer (which can take a day or two). Depending on the answer I may choose to open a bug - I could at this point open the test case and enter a bug and capture all the steps up to that point.....or If I decide not to open a bug I can continue with my execution.
There are many other reasons you may want to continue with a test case at a later time rather than have to start all over again.
I guess the short answer to "why this is necessary?" would be to "avoid unnecessary rework."
UPDATE: When I submitted this I saw an "Edit" button. Why is the "Edit" button necessary? To be able to continue with a post without having to start all over with my comments - same concept.
This feature has been impemented in the Visual Studio Update 1. You can go to http://www.microsoft.com/visualstudio/eng/downloads#d-visual-studio-2012-update to download this update.
Hi Anonymous, this capability has been implemented in VS 2012 Update#1 onwards.
View step attachments inline during execution, Add attachments and Pause/Resume tests in Web (http://tfs.visualstudio.com/en-us/news/2013-feb-11)
-Microsoft Product Team
We recently decided to switch to MTM for all future projects, but the absence of this feature has forced us to reverse our decision. There are many annoyances with MTM, but this one was the backbreaker.
At this very moment I am stuck with an In Progress test that I cannot resume. I had been working on it for an hour and a half. When I returned from my lunch break I found that MTM had crashed for some reason. The only option MTM offers is to start over.
I have been a QA professional for 10 years and this is the first manual testing tool I have ever encountered that doesn't allow you to save an incomplete or blocked test and resume it later.
Debra Lloyd-Forsyth commented
Anonymous as stated above this feature has been added to MTM 2012. You may want to re-evaluate your decision.
I am currently evaluating MTM 2010 & 2012 versions. The result of this evaluation will be me making a recommendation to my management team about which tool (if any) our test team should pursue. Having come from a HPQC background, I knew that evaluating other test products on the market, without having a biased opinion would be difficult. However, up until now I believe I have curbed any subjective opinions.
After reading several blogs and comments about this feature, it seems that this issue was spotted a couple of years ago in MTM 2010, and despite many people complaining, a supposed fix and a subsequent 2012 version being released, it has still not been resolved! But most shocking of all it seems that Microsoft do not grasp the importance of such a feature!
In my list of test product feature requirements, being able to pause and resume a test (without it being marked as failed!!) came up as a High requirement. Before we pursued our evaluation, we agreed as a team that if we came across any (high) requirement that was not satisfied - then the product under evaluation would not be recommended.
As it stands, because of this defect l will not be recommending MTM to my team. I say this with regret as there are many good features about the product, however this issue will ultimately be a show stopper to me and the rest of my team.
Please, please, please fix this issue!
The screen shots provided under 'more details' do not demonstrate the functionality requested. Does MTM 2012 provide the ability to Pause a test, and while it is still in pause status, start testing another test case and probably complete testing? If yes, could we please have have a post showing us how we can do that in MTM 2012?
This does not cover the request as there is already the functionality to Pause and Resume a tests in MTM. What the request is about is the ability to be in a test session, pause it for one reason or other and start a new test whilst the initial one is paused. So in reality not having to wait until one is finished before you can complete another.
Sahaswaranamam Subramanian commented
Visual Studio 2012 Update 1 CTP announcement says that Microsoft enhances MTM with a feature "The ability to pause and resume manual test sessions in Microsoft Test Manager".. does it cover this request?
This problem really DOES need fixing. It causes no end of problems and restricts testers from being able to pause a particular test (so that a bug can be fixed) and execute another test that is not affected by the bug. What use is it having to pause a test and then not be able to run any other tests?? Have you ever tried explaining to a Project Manager that you are unable to make any further progress on testing as you have had to pause a test until the bug pertaining to one particular test step has been fixed and you cannot run any more tests? Does it not frustrate you that the only options you have are to a) either have a quick scan through the test steps in the test case and BEFORE you execute it and only execute the test case after any bugs have been fixed b) execute the test, come across a bug towards the end of the test case and then have to run the entire test from scratch once the bug is fixed?? Hardly flexible and distinctly lacking when compared with other testing tools. I defy any experienced tester to say that this issue is not a problem.
This issue has caused us so many problems. Quality Centre and such like enable you to return to a script and run from the relevant step required, which is useful if you have raised bugs. This absolutley needs fixing and it concerns me that some people are saying that testers should not need that functionality - these are either people who have never tested before or testers that are not that experienced.
This isn't important to testers what they really need is less features and the new look of metro which is exactly what you get in 2012.
Beth Grayson commented
Yes, this functionality is absolutely needed in MTM.
Major design flaw that this functionality does not exist in MTM!
This is a big issue. In order for results to be useful this functionality must be fixed!
Still only 'UNDER REVIEW' - you had (possibly still have) the perfect opportunity to implement this critical feature in 2012... If you want to break into the testing space and take on HP et al - then you MUST listen to your audience and implement the features that customers (or potential customers) expect to see.
I know of 2 organisations who have been put off using MTM for this reason.
Paul Hachey commented
Unbelievable! What is the thought process that brings you to think that this ‘feature’ would not be considered a defect in your product? Add insult to injury, I now have to vote in order to have you fix your product. How many votes will it take and what is the projected time frame to fix this?
Roderick Bell commented
MTM has no way to stop a test case and resume from the last step
One process that we used a lot in Quality Center was the ability to start running a test case, pass a few test steps, stop execution and resume execution at a later time (either by the same or different person).
This functionality is lacking in MTM and the following Acceptance Criteria would fulfill that requirement.
1) User starts running a Test Case in the MTM Test module.
a) User does not modify any status - Run state stays Active or is not recorded.
b) User updates test step status and includes a Failure - Run state changes to Failed (even if there are steps not complete).
c) User updates test step status, but does not Pass all steps - Run state changes to Incomplete or In Progress.
d) User updates all test step status' to Passed - Run state changes to Passed.
3) If a test case run is in Incomplete or In Progress status, a user can choose to "Continue" running that test run or start a new execution.
a) Continuing a test run that is Incomplete or In Progress will carry over the status of the previously executed test steps and allow the user to execute the rest and either Pass or Fail a test run.
b) Starting a new test run will throw away the previous test run information and the test run is brought back to step 2.
Dan Gawarecki commented
Being "Agile" means a developer's priorities can be adjusted to commence work on whatever User Story is highest priority. Same principle applies in microcosm to running test cases. I may start test A but run into hardware problems and must pause it. Then start Test B but run out of time before leaving for the day, and want to pause it. The next day, my manager wants Test C run immediately due to a customer having a problem. MTM needs ability to pause any number of tests to be paused and resumed later, without any loss of artifacts from the results gained up to that point of pausing (i.e., all results are saved). I've used this capability in Quality Center for years, and wondering how MTM could be lacking such a key feature.
We are an MS Gold partner working with a large client at the moment and they cannot believe that this is missing - please implement.
Please implement this feature. We should be able to just "pick-up" from where we left off in a test case and not have to go through all of the steps!