DevTest Fail Test Case Companion is Not Working as Expected

Document ID : KB000115227
Last Modified Date : 18/09/2018
Show Technical Document Details
Issue:
Using Fail test case companion to fail the test case when any one of the assertions fails,  but it is not working as expected. It still shows all test cases as PASSED in DevTest Portal.
Resolution:
This is an expected behavior.  

The status "PASSED" for a test step means that the step has executed successfully but it doesn't mean that the functionality is passed or failed. DevTest records whether or not a given step has passed or failed, but this may not be an indicator of the business reason the step failed. 

Whenever there is an assert to fail, the test case will stop running immediately and then end in a fail state.  But when an assert is to error rather than fail, the specific step will be marked with error, the rest of the assertions and steps will keep running. The test will show it had "errors" but end in a PASS. However, if  the companion "Fail test case companion" is applied and if there are any errors in the run, the test will be marked as FAILED at the end. Generally in case of  Fail test case companion, "error" means DevTest is failing the test case but will continue with execution.

Ideally, this is correct because generally if a test step fails, DevTest will fail the test case and does not execute the next step. For example, if the first step of a test case is to login to an application and if it failed, DevTest will fail the test case and stop the execution. Even in ALM, if a test step fails, DevTest would fail the test case and stop the execution there itself. But since DevTest wants to run all of the assertions and keep running the test even if test step fails or had errorsDevTest will show that the test step had errors but status as PASSED. Also, the user can navigate to the step that has errors in the report and see the failure information by clicking the step.