Boxi Report Failiures - 500K

Document ID : KB000049538
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

Problem:

Multiple instances of BOXI reports spawned after daylight savings time change

  • Scheduled reports that are supposed to run once a day, run multiple times

  • Duplicate instances are being created

  • Instead of one instance failing/succeeding at the intended start time, ~650 instances fail per minute per recurring report

Environment:

  • BusinessObjects Enterprise XIR2 SP6 FP6.3 and below

  • BusinessObjects Enterprise XI3.1 SP3 through XI3.1 FP3.5

Cause:

This is a known issue with BOXI.

The BOE XI 3.1 FP3.x line has a defect that causes daily recurrences to be scheduled continuously on each day after the transition from Standard Time to Daylight Savings Time. This continuous scheduling will start from the time the recurrence becomes eligible to run and lasts for an hour. All instances produced for the recurrence during this hour will fail. The continuous scheduling will consume a large amount of CPU resources and will cause the CMS to become unresponsive. Applying this LAFix prior to the Daylight Savings Time change will prevent the problem from occurring.

Solution:

Resolution:

We strongly suggest installing BOXI SP4.

  1. For Business Objects Enterprise XI 3.1, this issue is resolved in the FixPack 3.6 and newer / Service Pack 4 and newer codestreams.

  2. For Business Objects Enterprise XI R2, this issue is resolved in the Fix Pack 6.4 codestream.

  3. If the DST issue has already occurred in the affected region, apply the the product-specific workaround below to repair the schedules.

Workaround:

The issue is related only to Daily recurring schedules. If you cannot upgrade to BOXI SP4, the reschedule script attached to this KB Article allows you to quickly resolve the problem and prevent it from continuing. This script is written by SAP (Oracle). Once fixed, you may need to cleanup the failed instances created during the issue. To do this, follow SAP note 1568718 (pasted below):

Symptom
I have 1000's of failed instances in my XI3.1 system. How can I delete these failed instances programmatically?

Resolution
Be sure to backup your CMS database & filestore prior to running this script. The script queries for all instances that have a SI_SCHEDULE_STATUS = 3 (3 being failed). It then loops through all the found instances and deletes them in batches of 100 objects. The script outputs to the browser the instances names and ids of what it is deleting.

Instructions:

  1. Download and extract the attached jsp into the "/<tomcat>/webapps/InfoViewApp/" directory.

  2. Edit the jsp using a text editor.

  3. At the top change the userid, password and cms name to be appropriate for your system. You can modify the batch deletion size by changing the 'int batchSize' value.

    String userName = "Administrator";
    String password = "Password123";
    String CMSName = "DEVR3SP3:6400";
    int batchSize = 1000;

  4. Save your changes.

  5. The script can be run by calling: http://<servername>:8080/InfoViewApp/deleteFailedInstancesV3.jsp

What this script will do:

  • it will run a query to display all your DAILY Recurring jobs

  • you will have the option to choose your jobs and reschedule them by 1 hour

  • you will then highlight the same job and change them back by 1 hour

  • by modifying the schedule times, you are modifying the schedule itself and this will keep the issue of duplicate instances from occurring.

Exact Steps to Complete:

  1. Stop the Job Servers.

  2. Double click on the .hta file.

  3. Change the System Name to that of your CMS name.

  4. Add the Administrator's password.

  5. Click Logon.

  6. Click List All Recurring Instances.

  7. Select All.

  8. Make note of what it says in: Schedule Selected Recurring Instances: Should be 1 hr earlier.

  9. Click Reschedule Selected Recurring Instances.

  10. Choose All instances again and change Schedule Selected Recurring Instances to 1 hour Later.

  11. Click Reschedule Selected Recurring Instances.

  12. Now start your Job Servers.

  13. The issue should not occur again.
File Attachments:
TEC567864.zip