Application Test Suite not working executing as per the defined values in the Staging Document

I am running a load test using the application test suite.  Using Load Pattern of Run N Times.

My test suite has 51 test cases and I have defined the following properties in the staging document : 

Instances        : 2 

Cycles             : 4 (Run Test Maximum Number of Times) 

Max Run Time : 5s (Maximum Run Time) 

However, the Test Suite only runs for 1 instance and 1 cycle and then stops. 

As per the documentation: 

Run N Times 

The Run N Times pattern applies when you run only one or only a few virtual users (instances), but you want to specify how many times the test runs. You are not concerned with any loading pattern. This pattern starts all virtual users simultaneously. 

To configure this pattern, enter the following parameters: 

Instances 

The number of virtual users. 

Cycles 

Select between running continuously for the interval set in Max Run Time, or running a maximum number of times. 

Max Run Time 

Select either No Max (the test case determines the time) or Maximum Run Time. In the latter case, specify the Maximum Run Time. This setting overwrites any value that is entered for Cycles. You can specify h for hours, m for minutes, s (or no letter) for seconds (default). 

So the Max time will override the Cycles, and since your max time was 5s, it only went 1 cycle for the 2 users. 

What you need for your load testing is to use Immediately Ramp in your Staging Document for the Load Pattern Selection. 

As per the documentation: 

Immediately Ramp 

The Immediately Ramp pattern applies when you are running only a few virtual users (instances) but you want to specify the test duration. You are not concerned with any loading pattern. This pattern starts all virtual users simultaneously. 

To configure this pattern, enter the following parameters: 

Instances 

The number of virtual users. 

Max Run Time 

Select either No Max (the test case determines the time) or Maximum Run Time. In the latter case, specify the Maximum Run Time. This setting overwrites any value that is entered for Cycles. You can specify h for hours, m for minutes, s (or no letter) for seconds (default). 

So if you choose Instances of 2 and a Maximum Run Time of 5m (5 minutes), it will show 2 virtual users on the Reporting Dashboard, this will run for 5 minutes and it will also show how many Cycles executed for each virtual user. When your Suite finishes, go to the DevTest Console and choose Reporting. Then click the Grid checkbox true on the upper right side of the page. Then double-click your Suite, it will show the test case. Double-click on the test case to see the detail. 

Refer to the Load Pattern Selection section in the DevTest document for the release you are using.