enable changing the order of test cases on the web gui and let them be tested in this order for manual tests
With the MTM 2012, it is possible to change the order of test cases within one test suite and this order is considered when testing. This functionality is missing in the web environment. It is not possible to order test cases and then have somebody else test them in a specific order. If a test suite is run, the tests are ordered by ID only. Make this work for manual tests.
In Test Manager the user has the option to have an 'Order' column, however when I go to TFS 'Test'/Column Options there is no order option.
When I have manual test suites created in MTM I can designate an "Order" for the tests. If I execute (run) the tests in MTM, it runs in the order that is associated to the suite. When I have users log into the Team Web Access site to execute the manual tests, the tests do not execute in the order set in MTM, they execute according to the ID associated to the test case.
In MTM 2012 you are unable to change the order of test cases in a suite that is requirement-based. In a normal test suite it is possible, but it would be great if we could change the order also in requirement based test suites.
MTM 2012 doesn't support ordering tests that are created from Query Based or Requirement based test suite. It would be good to have this feature as I am trying to Link the Automation with MTM tests.
So far I have used Order Test from Coded UI Solution file for triggering Automation, But now when I am trying to do it from MTM I don't see an option to Order the tests for Requirement based and Query based testsuites.
Need to be able to Run Tests in Suite from Command Line in Order in MTM.
Using below command im able to execute automated test case in a test plan(MTM)
"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\TCM.exe" ru
n /create "/title:XXAutomation" /planid:17 /suiteid:528 /configid:3 /co
ngsname:Automation default" /builddir:C:\Share\xx\XX_20130515.3 "/test
These test cases are executing based on the test ID(ascending) not by order of test cases in MTM.
It would be good if you provide a option to run test cases in a test suite according to MTM order
When you need to update the order of a high number of test cases (20+) current workflow is tedious. If you try to update multiple order numbers at a time, it is confusing because sometimes you don't get the expected order. You should enable drag & drop of test cases to expected order.
Currently only test cases in a test suite that is not a query-based test suite or created from a requirement or user story can be reordered.
What's even worse is that in MTM in the Plan Stage it appears as if you CAN reorder them but when you move into the Test Stage every test in the Suite is reordered by ID - this is infuriating and makes a complete nonsense of test cases that must be followed in sequence.
The current Order feature in Dev11 for reordering test cases in a test suite is clunky. Instead of navigating away from the Contents page, I'd like to have the ability to reorder TCs by dragging/dropping them directly in the TC list of the suite.
If you have a test plan in MTM and you begin to automate tests within that plan you don't have the option of running those automated tests in the order in which they were originally defined. The "order" field doesn't apply when in reality it should make the tests into an "ordered" test.
Unlike Static test suites, you cannot currently specify an order for test cases created under a specification based suite. Would be useful!
We have enabled support, on Team Services, for ordering of manual tests across static, requirement-based and query-based suites, directly from the Test hub. This functionality will become available for OnPrem with TFS 2016.
You can read more about this from here:
Marking this item as “Completed”.
Note: If you need the ordering support for automated tests, please vote on this uservoice item:
Hi, nice post. I wanted to know if we can find a solution to run automated test cases in a particular specified order by us.
I tried the following options but it didn't work:
• Created a new suit and re-ordered the test IDs(by changing the order and clicking on "apply changes") and gave run.
• Checked all the column names values(such as 'activated date','start date','automated test id'etc.) to see if there is a pattern by which MTM picks the test case that is to be run next. Found none.
• Created another suit and added each test case one by one. Gave run and found that they also didn’t follow the order in which they were added in the suit.
It would be great if you could help me with it. Thanks.
Problem: i have added test cases in Static suite but i want to execute them in some other order , So any one can help me out that how I can re arrange them in static suite using MTM or web TFS
Can it be possible on Static suite
Grégory Ott commented
Totally agree with @Shubham Srivastava. The ordering of test cases should work with Query Base Suites and Requirement Base Suites
Emmanuel Jouan commented
This can be temporarily be fixed by using the "Create copy and Add to Suite" in the required order and then deleting the original test cases that were in the wrong order as the newly copied/pasted will have new incremental IDs
how can i order test cases in a suite?
I'm with the same problem in my team since 2012. I really would like to have users testing the system with test cases using web test manager, but it´s impossible without the correct order!
Shubham Srivastava commented
Also my observation is that "Order" works only for Static Test Suites and not available for Query based suites and Requirement based suites . I think Test Manager and Visual Studio Online both should have the option to do reordering for any type of suite.
I can only agree with John Public. I am curious that have the product team here ever done real automated testing before?
John Public commented
This is sorely missing from web view and rather sad that after bringing this up over two years ago and receiving that many votes all we get is an 'under review'. This feature is direly needed.
I disagree that automated tests should not have dependencies. I understand the motivation for that statement, but testing a few dozen variations of a complex report is way faster when allowing for dependencies because each test case only has to do minor tweaks to data rather than wipe everything clean and spend extensive amounts of time to enter data differently. That is a total waste of resources even when a machine executes the tests. Same with tests that hit services that charge per hit (some geolocation services do that). Preventing unnecessary service calls not only saves time, but also a lot of money!
Besides that, ordering tests in web view or anywhere else as no impact on independent automated tests. They will run in the set order, but they can run in any order and yield the same effect. So picking just one order will be perfectly fine.
The more I think about it the more it seems as if this is just some statement to have a bogus reason not to implement this feature. Please prove me wrong!
Paolo Sotgiu commented
The ordered execution of test is a must have for us in the scope of an automated execution of the regression test. I can't really understand what is the point in supporting it for manual tests and not for automated test (talking about coded UI)...the setup for some tests in our cases will require to duplicate a lot of the logic we already have in other test methods which are normally executed during the script (test the record creation, then the edit and then the removal....)....we will triplicate the lenght of tests and they already require 2 days to complete. At least give us the option to decide to use the ordering or not, as it is available already in MTM. I really miss NUNIT right now....
Gerhard Matzen commented
I do not like to use ordered tests for automated tests. This is because the ordered tests is reported as a monolithic pass / fail in MTM. I would much rather see individual test results instead of a monolithic pass / fail.
Sascha Kolb commented
If you really want to do this only for the Manual Testcases, plesae split this idea and see where the votes go. Thanks
Sascha Kolb commented
I dislike the way this Feature is going.
I prrincipal agree that,
"For automated test cases, the best practice is not to have dependencies between the automated tests"
But, there is options, where this is not feasible, or expensive havin each case Independent.
We e.g. use Testcases also ón a System Level with a mechatronic System, where bringing it to a inital state (rebooting) is expensive, thats why our AUTOMATIC testcases are dependend. But we want quick Feedback. Thats a trade off.
I would really see a Option to Order ALL Testcases in future also via web.
any workaround to this?
ahora necesito conocer cómo se llama y dónde está ese campo "order" en SQL 2008 para imprimir un reporte... alguien sabe??? gracias
Yes... but in spanish...
desde el Test Plan hay que crear una Suite, entonces adentro de la Suite vinculás el Requirement... luego, copiás los Test Cases del Requirement sobre la Suite...
Te posicionás sobre la Suite y arriba a la derecha vas a ver un botón llamado "Order". Hacés click en ese botón, y listo... configurás el orden de los casos de prueba para esa ejecución o las posteriores.
Dee Huskey commented
is there any plan to add the ability to order tests on the web, and if so when? Has a release been scheduled?
Need this now
Lucy Goddard commented
This is critical for us !