CA IDMS CV Journal Sizing

Document ID : KB000050631
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

The journal sizing considerations presented should be reviewed whenever the processing characteristics of a CA IDMS CV change. The change may be due to the addition of more users to the environment or the addition/modification/deletion of applications controlled by the CV. Pro-active monitoring will help to insure that a CV's journal environment maximizes the available resources and minimizes journal I/O and the potential associated processing bottlenecks.

Solution:

Within a Central Version (CV) journal activity can be a hidden yet significant processing bottleneck for update transactions. When CA IDMS updates a page all journal blocks containing images associated with that page must be successfully written before an attempt is made to write the database page. Therefore the number of journal blocks needed to contain a page's updates can significantly affect the amount of time a transaction must wait before a database page is written and the transaction can continue processing. This article will attempt to help you minimize this bottleneck by addressing the 3 major journal sizing considerations for a CV.

  • Journal block size
  • Journal transaction level
  • Journal Buffer size

Journal Block Size

A goal of the DBA is to minimize the number of journal blocks needed to contain a page's updates thus reducing the amount of time a transaction needs to wait for journal I/O. Unfortunately there is no straight-forward formula that can be used to estimate the ideal journal block size for a CV. There are numerous types of records written to the journals based on the types of processes that occur within a CV and each type may have a unique length. In addition a single journal block has the potential to contain records generated from multiple transactions making it very difficult to predict the contents of a typical block.

The most numerous types of records found within a journal are BFOR and AFTR records. These records reflect the contents of a database record before an update operation (BFOR) and the contents following the update (AFTR). The lengths of these records consist of 64 bytes of overhead and the length of the data record. The length of the data record should be calculated as the length of the record's prefix and the data length. The most significant recommendation regarding journal block size is to make it large enough to hold a BFOR and an AFTR image of your largest data record so that a single update might be contained on a single block although this can never be guaranteed due to the possible presence of other record types already existing on the block. However having a journal block smaller than this value will guarantee at least two journal blocks must be used to record the update thereby necessitating two writes to the journal file.

Once a journal block size is selected the output from a number of ARCHIVE JOURNAL jobs should be reviewed to determine the general percentage of space used on a journal file's blocks. Figure 1 shows a sample space utilization histogram produced by the ARCHIVE JOURNAL utility.

Figure 1

Figure 1

If the majority of pages fall into the higher utilization range of the histogram it is probably an indication that the selected block size for the journal file is too small and should be adjusted so that there is mix of page space usages with the average page usage in the 70 to 80% range.

Journal Transaction Level

The downside of selecting a larger block size for the journal files is that partially filled blocks waste disk space and increase the number of times the CV journals fill in the course of a day resulting in more frequent journal swaps. An increased number of swaps will also mean that more journal archive files are created which will result in more files having to be maintained and processed during manual recovery operations.

To maximize the space in a journal block CA IDMS provides a means to defer writing journal blocks called JOURNAL TRANSACTION LEVEL. The purpose of deferring the journal write is to enable additional images for other update transactions to be placed in the journal block before it is written, hopefully resulting in full or nearly full journal blocks. When this is achieved fewer journal writes are issued, disk space is better utilized, and fewer journal swaps occur.

Journal transaction level can be set by specifying the parameter on the sysgen SYSTEM statement or by issuing the DCMT VARY JOURNAL TRANSACTION LEVEL command. The value set should be either a 0 which disables the functionality or a value of 3 or larger. Setting a value that is too small or too large may prevent a CV from getting the full benefit of the function or may completely eliminate any potential benefit. As long as the number of active update transactions in the CV exceeds the specified value, the write of a journal block is deferred until the block is full.

After establishing or changing a journal transaction level the space utilization histogram from several ARCHIVE JOURNAL runs should be reviewed to see if the change has had an impact and whether that impact is positive or negative. The average response time of update transactions should also be monitored. When a transaction's journal write is deferred that transaction is placed in a wait state until the journal block is filled and the I/O completed. If there are sufficient update transactions active but the amount of updating is low it is possible that the use of a journal transaction level may increase the amount of wait time for a transaction to levels greater than those experienced without the feature. In those cases it may be more desirable to accept a less efficient usage of disk space for faster throughput.

The graph in Figure 2 shows the number of journal blocks that were written to perform 20000 debit/credit transactions at varying journal transaction levels. In this example the most benefit was derived by setting the journal transaction level to 4. More than 50% fewer journal blocks were written to perform the same volume of work compared to what was required when the journal transaction level was set to 0 or 10. A value of 0 disabled the journal transaction level while a value of 10 resulted in an environment where there were rarely enough active update transactions to cause the writing of journal blocks to be deferred.

Figure 2

Figure 2

The chart in Figure 3 illustrates the effect that Journal Transaction Level has on how full the journal blocks were when written. With a journal transaction level of 4, about 50% of the journal blocks were almost or completely full versus 0% when the journal transaction level was set to a value of 0 or 10. Conversely, with a journal transaction level of 4, less than 20% of the blocks had a utilization of less than 10% versus about 50% of the blocks with a journal transaction level of 0 or 10.

Figure 3

Figure 3

Journal Buffer Size

To allow for journal images to be created while earlier blocks are being written a pool of buffer pages is maintained by CA IDMS for the journal environment. Unlike a database buffer pool which is accessed randomly, the journal buffer pool is accessed circularly. When a journal buffer needs to be written its I/O is started and the next journal buffer becomes active. This reduces or eliminates waits for a journal buffer when a write to a journal file is required. When the I/O is completed the buffer remains as is until it becomes active again and the previous journal images are overlaid. Multiple journal buffer pages may also reduce journal I/O during recovery by allowing a BFOR image to reside in memory longer. If a recovery operation must read in a journal page in order to access a BFOR image, it will look in the non-active buffers for the BFOR image before accessing the journal.

A journal buffer should minimally contain 5 pages and may benefit from many more pages in active systems that incur many abends that lead to database recovery. The adequacy of the journal buffer pool can be monitored by issuing the DCMT DISPLAY BUFFER command.

Figure 4

Figure 4

In almost every CV the most important statistic in Figure 4 is the number of waits. The number of waits should always be zero. When an I/O is started on a journal buffer the buffer is flagged as unavailable and the next buffer becomes the active journal buffer. Upon completion of the I/O the buffer being written again becomes available. Many journal buffers may be unavailable at any one time because they are all concurrently in an I/O state. If all of the buffers in the journal buffer pool are unavailable the next request for a journal buffer will add to the wait count. Therefore if the wait count is non-zero it means that all of the buffers in the pool were in a wait state at the same time and all transactions had to wait for a journal buffer to perform an update operation.

When the wait count is greater than zero the number of pages within the journal buffer should be increased until this number is once again at zero. Increases in the pool size should be made in small increments as a small change can have a visible change in the number of waits encountered.

In a few CVs that experience high numbers of recovery operations there may be some value in defining a large journal buffer pool. The '# of Recoveries' in Figure 4 represents the number of aborted transactions that had to roll out updates and 'I/Os' are the number of journal blocks that were read to perform the recoveries. The 'in Buffer' value is the number of journal blocks that were found within the journal buffers and therefore did not require an I/O operation to be performed.

Recovery will not use the CV's journal buffer pool to read the journal but maintains its own buffer consisting of a single page. Recovery will search the CV's journal buffer pool to locate a required BFOR image and if found copies that buffer's contents into its own buffer causing the 'in Buffer' value to be incremented. If a CV is very active and is writing journal images at a high rate the BFOR images for an aborted transaction may not be in the journal buffers by the time the recovery starts. This is especially true for abends resulting from deadlocks and timeouts. If the CV does a large number of recovery operations increasing the journal buffer pool to a large number such as 300 or 400 may result in an increase in the value of the 'in Buffer' statistic indicating a more efficient recovery environment. However if the 'in Buffer' value remains small after making the increase the enlarged buffer pool is probably just wasting storage space and should be reduced to the minimum level where journal waits will be maintained as zero.

Summary

The journal sizing considerations presented should be reviewed whenever the processing characteristics of a CA IDMS CV change. The change may be due to the addition of more users to the environment or the addition/modification/deletion of applications controlled by the CV. Pro-active monitoring will help to insure that a CV's journal environment maximizes the available resources and minimizes journal I/O and the potential associated processing bottlenecks.