Test cases use data sets in the workflow and sometimes Test execution doesn't show all the data set rows and it looks like data sets are being read out of sequence.
When Test is run in ITR, we see the iteration go through all the data set rows but when test is staged or run using TestRunner and through Jenkins not seeing data. For Example:
* Staging the test in Workstation:
Stage Window shows only few rows of data when the data set has around 20 rows or above. If we select "No Filter" option and select the events to show specific data set information then the execution of all 20 rows is seen and reports from DevTest Portal also show that.
* Run the Test using TestRunner and create a HTML report:
The test runs fine but the HTML report skip few rows and show data for some rows only when data set has around 20 rows or above. When data set has fewer rows then the report shows (12 or so) fine.
DevTest limits the number of events that will write in the html file to 32 (lisa.CycleExecHistory.buffer.size=32), which is why the report is truncated. The reason you can see this in the portal is that it is collecting the information from the database. This is why the number of events needed to be filtered in the Workstation to see what was wanted when staged there.
Why DevTest does not show all the data set rows in test execution?
All supported DevTest Environments.
. To see all the data set rows in Workstation-->Stage window, need to tweak the filter option. Recommended to add the "Dataset Read" event to the filter, which should show all data set rows.
. To see in the HTML report through TestRunner, set the property lisa.CycleExecHistory.buffer.size to a higher value than 32 (default) in Simulator's vmoptions files. If you running TestRunner using default Simulator(embedded) then configure the property lisa.CycleExecHistory.buffer.size in TestRunner.vmoptions.