The step status which is "PASS" becomes different status at the "Test execution detail" report in the DevTest Portal.

Document ID : KB000008912
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

If a test case has multiple steps and the test result has steps which statuses are "PASS" and only one step which status is "ABORT" or "FAIL", then the "Test execution detail" report incorrectly shows that all the steps are "ABORT" or "FAIL".

Here is an example that reproduces this problem.

  1. Start Enterprise Dashboard, Registry, Broker, DevTest Portal and DevTest Workstation
     
  2. In DevTest Workstation, create a new project or open an existing project
     
  3. In the project, create one test case which includes 4 "Execute script(JSR-223)" steps.  And the 4  "Execute script(JSR-223)" steps include the same code only:
    return 1;
    by using beanshell like below:


  4. Add one assertion which will "ABORT" the test if the step result(property named LASTRESPONSE) equals "1" to the third "Execute script(JSR-223)" step like below:

    and save the test case. Then this test case is always aborted after third "Execute script(JSR-223)" step.

  5. Run this test case with Stage Test
     
  6. After running this test case with Stage Test, then the test result which can be seen in DevTest Workstation like below:

The information is displayed that the first and second steps are "pass"(green characters) the third step is "abort".

And the Monitor Test in DevTest Portal displays the same test result like below:

The first and second steps are "Passed", and the third step is "Aborted", this information is the same information which is shown in DevTest Workstation.

However, the "Test execution detail" report in DevTest Portal displays an incorrect 'Step Status' result like below:

The statuses for all the steps are "ABORT".

If the the next step information of the assertion in the third "Execute script(JSR-223)" step is changed from "ABORT" to "FAIL", then the "Test execution detail" report displays same test result like below:

The statuses for all the steps are "FAIL".

Environment:
DevTest 9.1, 9.5, 9.5.1, 10.0 and 10.1
Cause:

This is a known problem for the "Test execution detail" report in DevTest 9.1, 9.5, 9.5.1, 10.0 and 10.1.
This problem is solved in DevTest 10.2.

Resolution:

There are no workarounds in DevTest 10.1 or lower.
An upgrade to DevTest 10.2 is one of the solutions.
Please contact CA Support if using earlier releases.