Tuning the Host Database
CA Workflow stores its objects in a database, and databases often need to be tuned for optimal performance. This is true for the CA Workflow schema.
Embedding products that use the CA Management Database (MDB) need less tuning than those that create a standalone CA Workflow schema. In particular, the MDB sets crucial page sizes correctly and tunes the DMF cache in Ingres.
Because CA Workflow relies on the correctness of its schema, which is stored in the underlying database, incorrect Ingres tuning can cause problems. Some of these problems are:
- Performance problems and hangs due to unnecessary deadlocks or disk thrashing.
- Inconsistencies in XML data stored in BLOB columns. Ingres does not detect these data inconsistencies, and CA Workflow may not operate correctly after they occur.
- Workflow objects that do not appear until the IDE or the Worklist is refreshed.
Tuning Ingres is a large topic. For sake of CA Workflow:
Use of checkpointing instead of journaling
- Be sure you have the correct maintenance installed
- 2.6 SP2: Patch p11298 for Double Byte or later
- R3: r3.0.3 build 103 (Windows) or r3.0.3 build 211 (Linux or Unix) or later with current maintenance applied
- Meet the requirements for minimum tuning found in SupportConnect solution QI74334 - RECOMMENDED INGRES CONFIGURATION SETTINGS:
- Page sizes - use queries from QI75852, CORRUPTION OF PROCESS DEFINITIONS USING INGRES 2K PAGES, to verify correct page sizes after installation.
- DMF cache enablement and sizing. DMF cache tuning is absolutely critical for performance. Without it, normal page deadlocking may escalate to table locking and lead to system problems that are hard to diagnose.
- Other general recommendations, especially those such as isolation and locking.
- QI75951 WORKFLOW WITHOUT MDB - INGRES BACKUP AND RECOVERY, which points to this Tech Doc
- QI81614 - INGRES JOURNALING NOT RECOMMENDED FOR WORKFLOW SCHEMA
- Keep your CA Workflow schema purged using the Workflow Development Environment (IDE). This requires Workflow r1.0.019.011 or later to work well.
- Reorganize your schema as needed to prevent stringy b-trees.
In addition, depending on the needs of each particular site, advanced tuning may be required, but that is beyond the scope of the current document.
SQL Server Tuning
When the Workflow schema is persisted in a SQL Server database on a machine with more than 2GB of RAM, tune SQL Server according to this Microsoft knowledge base item to take advantage of all the available memory:
See also these informational solutions:
- QI83103 - INSTANCES TABLE GROWS RAPIDLY ON SQL SERVER
- QI86805 - SQL SERVER UNUSED ALLOCATION
When the Workflow schema is persisted in an Oracle 10g database, these server parameters should be tuned for optimal performance:
This is the recommended SQLPlus session, logged in as SYSDBA:
SQL> ALTER SYSTEM SET open_cursors=1500 scope=both;
SQL> ALTER SYSTEM SET db_file_multiblock_read_count=n scope=both;
SQL> ALTER SYSTEM SET processes=300 scope=spfile;
When complete, you must restart Oracle.
The value of db_file_multiblock_read_count should be chosen carefully. A value of 32 for n is a good place to start, but the operating system on which Oracle is running must be capable of reading db_file_multiblock_read_count * db_block_size in a single I/O request. If it is set too high, the optimizer will think that full table scans are cheap and will prefer them to the more efficient use of indexes. On the other hand, setting it too low makes the optimizer choose indexes more often than necessary.
This parameter specifies how many blocks will be read at once when Oracle performs a full table scan or an index range scan. It doesn't affect reads on blocks that are indexed, in which case only one block is read.
App Server Tuning - Tomcat
The Tomcat servlet container offers several settings that can be adjusted to maximize throughput of CA Workflow (shown below with current Unicenter Service Management defaults). High values use more memory.
- minProcessors=5 is the minimum number of threads Tomcat keeps available for use in processing incoming requests for service
- maxProcessors=75 is the maximum number of threads Tomcat will make available for use in processing incoming requests for service
- acceptCount=100 is the number of unprocessed incoming requests that will remain queued before Tomcat starts rejecting requests for service.
A request for service in CA Workflow terminology means a web service request coming from the outside world or CA Workflow itself is requesting activity. Once acceptCount is exceeded, then the requester gets a "Connection refused" error returned.
To set Tomcat 4.1's memory usage parameters, change the CATALINA_OPTS environment variable and restart Tomcat. For example, to set the maximum Java heap on Windows with an initial allocation of 256MB:
set CATALINA_OPTS=%CATALINA_OPTS% -server -Xms256m -Xmx1024m
Some of the embedding products parameterize Java heap parameters. For example, Unicenter Service Management provides these parameters in fulfillmentService.conf:
- initmemory= 128 is the amount of memory in megabytes initially made available as the Java heap size
- maxmemory= 512 is the maximum amount of memory in megabytes allocated to the Java heap
On the other hand, Unicenter Service Desk currently hard-wires the maximum heap at 512MB. To adjust this setting, see TEC418959.
With Windows, the maximum Java heap size is between 1 GB (1024 MB) and 1? GB. to determine the maximum heap that can be allocated by the machine:
Run java -Xmx nnnn m
Keep bumping up nnnn until the java invocation fails like this:
C:\ >java -Xmx2000m
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
and then back off to your previous setting.
This limits other memory-intensive settings for Windows, regardless of the amount of RAM on the machine.
On Unix or Linux, the Java heap is limited by available physical memory.
For more information on tuning, see Sun's Java Tuning White Paper.
Note: You should specify a value less than the amount of RAM in your system, so the operating system and non-Java applications will have enough memory to run. If you don't, the OS will use its disk-based swap space and overall performance will degrade, often severely. Some sources recommend allocating 80% of a machine's RAM to the Java heap, but this assumes almost complete dedication of the machine to the use of the JVM.
For comprehensive Tomcat tuning options, please see http://tomcat.apache.org/tomcat-4.1-doc/config/index.html .
Useful external links on Tomcat tuning:
App Server Context - Connection Pool Use
The number of database connections available to CA Workflow is set in the pm.xml and wl.xml files that configure the Process Manager and Worklist contexts:
- maxIdle (default 30) is the number of connections that will be in the pool even if none are being used. Preallocate a larger number if you expect to hit the server with many simultaneous process instances immediately upon start-up.
- maxActive (default 100) is the maximum number of connections that will be obtained before errors result. Set maxActive to provide at least two database connections for each engine in the EnginePools setting below.
There are several CA Workflow adjustments that can be made using the IDE Server Configuration UI:
- EnginePools (default 50) is the number of Process Manager engines available to handle process instance activity. The number of process instances that can be executing concurrently is determined by the number of process engines available within the Process Manager server. Note: The number of instances that exist in the Workflow server is not limited by this number, since an instance gives up its engine from time to time during its execution.
Raising this number may help with throughput and reduce errors in starting process instances, but it may also consume large amounts of memory since each engine holds a process definition. When increasing EnginePools, you should also increase the Tomcat connection pool.
- MaxWorkItems (default 2500) is the number of workitems per process instance that are kept in memory before being stored on disk. It generally needs to be increased only if a process definition is very large or includes a very large number of iterations.
- TimeOut (default 3000) is the number of milliseconds (3 seconds) that the Process Manager waits for an engine to become available before failing requests for work to be done (related to the number of engines specified in EnginePools). A Worklist UI user will get a "Process Engine Busy" message and may just repeat what he was attempting. An API caller will get a "Process Exception", and so should be able to handle this exception and retry if that is appropriate.
- WebServiceTimeout (default 600) is the number of seconds (10 minutes) a web service activity in a process instance will wait for a response from the called server before declaring an actor fault. Reducing this timeout may cause service requests to fail that would otherwise succeed. Increasing it slows the responsiveness of the server to web service failures.
CA Workflow r1.1 provides clustering support for load balancing. It also supports Tomcat 5.0, WebSphere, WebLogic, and JBoss.