I suggest you ...

Coded UI - Have a "continue on error" for asserts

Currenlty if a assert fails, the test stops
(Since the CodedUI is built on top the unit test framework)

But in a Coded UI test you may have multiple assert in a page and currently you have code a Try Catch block.
In web performance test the continue on error does exist.

(the feature pack 2 as misleading Continue on Error but that is only associated to playback)

90 votes
Vote
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    I agree to the terms of service
    Signed in as (Sign out)
    You have left! (?) (thinking…)
    Anonymous shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    7 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      I agree to the terms of service
      Signed in as (Sign out)
      Submitting...
      • Anonymous commented  ·   ·  Flag as inappropriate

        This would be really helpful. With UI tests, the process of getting the application to a particular state to test something can be time consuming. It doesn't make sense to apply the unit test methodology of test one thing and one thing only when the test can take minutes to run and there are lots of values to test.

      • Sudhish MathuriaSudhish Mathuria commented  ·   ·  Flag as inappropriate

        I think a parameter will serve the purpose. We can suggest playback engine with this parameter that it needs to fail or continue the test. We can keep this parameter as optional and by default value will be fail.

      • Grey HamGrey Ham commented  ·   ·  Flag as inappropriate

        I think the concept of "Verify" semantics are required - similar to what is used in GTEST. Verify() is exactly the same as an Assert() except that Verify() does not terminate the test on a failure - it 'notes' the error but continues.

        At the end of the test, if any Verify's failed, the test is failed (warnings tend to be ignored). This approach allows a lot more information to be gleamed from a failure, especially where MsTest is being used for, perhaps, functional UI testing and the test might contain many assertions.

      • Anonymous commented  ·   ·  Flag as inappropriate

        Well that would be better than what is currently, it would help in test desing and it could be easily discoverable in the assert's property windows (it using the feature pack add-in)

        But I would still prefer my test to fail. Here is why, by experience, warnings are looked at as seriously, for example code analysis are often "warnings".

        I think the team takes it more seriously when it "fails".

        Also, I wonder what trickle effect such a new status would have on team build (would team build still be "partially succeeded" and the build parameter "fail build upon test failure" check box?)

      • Mathew Aniyan(MSFT)Mathew Aniyan(MSFT) commented  ·   ·  Flag as inappropriate

        Will an overload to the Assert Methods which specify that an assertion failure should be a warning suffice?
        By default an assertion failure results in a test case failure. If this new Assert overload method is used, a warning will be logged in the test results.

      Feedback and Knowledge Base