Task Persistence Database: Use, Performance, and Troubleshooting:
- Internal task and event state tracking until task completion.
- Stored in a database with the intention immediately deleting the Task Persistence data when the task completes.
- Writes huge number of rows and performs high r/w activity to DB by design, therefore should be cleaned up promptly and frequently.
- User viewable via View Submitted Tasks for verifying task completion and for troubleshooting assistance.
§View Submitted Tasks (VST) provides insight into the Task Persistence (TP) data as a simple way to monitor recent task completion/failure.
Not intended for use as:
- For audit, reporting and compliance use IDMs reporting and audit functionality, do not use task persistence data
- Not intended to be accessed by custom code for the following reasons:
§ Can interfere with performance
§ Can cause DB lock conditions
§ Avoid writing custom code or PX policies that access TP database
- Every task and event of each task have states that change from one processing step to the next, each of which is tracked in TP. This requires many DB connections and high r/w activity for each task.
- The larger the TP database grows, the longer the queries take, which can significantly impact system performance.
- A large TP database with heavy activity can become a performance bottleneck in which tasks appear to be processing slowly or seem to remain ‘in-progress.’
Database Space Usage
You can minimize the amount of data in TP tables using the following methodologies:
- Run Cleanup Submitted Tasks
§ Use for ongoing cleanup
§ Not effective for massive cleanup
§ Run daily or weekly keeping only most recent few days of data
§ Removes only Completed tasks—no risk of removing in-progress tasks
- Keep only most recent tasks
- For mass cleanup, run stored procedure, “garbageCollectTaskPersistence”
§ Stored procedure removes only completed tasks, not in-progress tasks.
§ Stored proc is installed into Object Store
§ Avoid running during busy hours
§ Can take a long time—run small batches
§ See the Readme and SQL Script for running the stored procedure in
..\CA\Identity Manager\IAM Suite\Identity Manager\tools\samples\TaskPersistence\SQLServer
§ See KB000043199, Cleanup Submitted Tasks Not Working; Database stored procedure for cleaning up task persistence is available.
Stuck In-progress Tasks
- Can be slow and not progressing due to bottleneck
- o Or permanently stuck due to task state inconsistency (system failure, for example)
- o Tasks stuck in progress can be resubmitted with the Post-Events feature (available in IDM12.6.8 or later)
§ The Post-Events feature allows the user to resubmit events that get stuck in-progress. This feature is accessible via the Post Events button on the Identity Manager Environment Properties page.
§ See Post Events to the Identity Manager Event Service in https://docops.ca.com/ca-identity-manager/14-2/EN/configuring/ca-identity-manager-environments/manage-environments#ManageEnvironments-PostEventstotheIdentityManagerEventService
- Tasks sometimes become stuck due to inconsistent state and corruption cannot be restarted; cannot be cleared via View Submitted Tasks
§ It’s OK to just leave them as is.
Diagnosing Stuck In-Progress Tasks
o Usually due to a processing bottleneck
§ Bulk feed processing?
§ Explore Correlate running?
§ Check VST, look for patterns of stuck tasks
§ What is the last event (password sync to prov, for ex.)
§ Check for Task Persistence DB slowness
§ User directory or Object store DB connection issues
§ Check IDM application server logs for problems
§ Provisioning server issues
· Check Prov logs (etatrans, etanotify, JCS)
· Inaccessible endpoints
· Connector server not running
· CA directory errors
· Outbound sync (to IDM) failures to ETACallback servlet
· Notifications not processing, or not keeping up with load
§ Check Java Messaging Service (JMS) processing for problems