Rebuilding a Data File While Eliminating Corrupt Data

Document ID : KB000065858
Last Modified Date : 27/08/2018
Show Technical Document Details
Introduction:

Although the best solution to deal with corrupt data is to restore a copy of the database taken prior to the corruption, this is simply not always possible. Backups should be taken frequently with the CA Ramis BACKUP utility. This will ensure a good copy of the database is available should a problem occur. So what can you do if you don't have a good backup of a corrupted file? Several methods of determining corruption, salvaging data and rebuilding a file may be appropriate depending upon the file structure and the location of the bad data. This article provides one simple example of how to pinpoint corrupt data and how to rebuild a data file when a backup is unavailable.

 

Environment:
z/OS
Instructions:

Once it has been determined that a file is corrupted, you will want to pinpoint the location of the corrupt data to allow you to retain as much of the good data as possible. Generally, if a file has been corrupted, an error will be encountered when accessing the file. (The error could be an abend or any of the following messages: SY0020, SY0021, SY0025, SY0026, SY0030, SY0130, SY0140, ...)

In this example, let's assume that we have discovered that our sample file PRODINFO is corrupted. PRODINFO is a 2-level file with 8 fields. We can display this information by issuing the command RAMINDEX PRODINFO and reviewing the output.

DESCRIPTION OF CA-RAMIS FILE: PRODINFO                PAGE     1

L E V E L  SEGMENT
LIST   FIELDNAME     SYNONYM       NAME TYPE  FACTOR   FORMAT
----   ---------     -------       ---------  -------  ------
1  PRODNUM       PN            001  D        1     A   6
2  PRODUCT       PNAME         001  D        1     A   20
3  PRODTYPE      PTYPE         001  D        1     A   11
4  DISCOUNT      DCLASS        001  D        1     A   2
5  MONTH         YM            002  S       12     I   4
6  LISTPRICE     LIST          002  S       12     P   10.2
7  STDCOST       UCOST         002  S       12     P   10.2
8  PLANQTY       PLAN          002  S       12     I   6

The first thing we need to identify is where the corruption exists. To begin, we will issue several TABLEF requests beginning with the first level.

TABLEF
FILE PRODINFO
PRINT PRODNUM AND PRODUCT AND PRODTYPE AND DISCOUNT
END

This will cause CA Ramis to read every record on the first level of the file. If you get a message indicating there is an error, we know the corruption exists on Level 1. Since this is a multiple level file, if no error occurred on Level 1, we would continue to Level 2 and run a similar request referencing the fields on that level.

TABLEF
FILE PRODINFO
PRINT MONTH AND LISTPRICE AND STDCOST AND PLANQTY
END

Once we have identified the Level on which the corruption exists, we can try to isolate the damaged data. We determine what records are affected by including a selection statement (IF or WHERE) in our TABLEF request. Our goal is to be able to test a value greater than and less than the corrupted data. For example, we can see in the RAMINDEX output that PRODNUM is defined as an alpha field with a length of 6. If the corruption exists on Level 1, we might choose to start testing the PRODNUM field at the mid-range of " L "s by using the following request:

TABLEF
FILE PRODINFO
PRINT PRODNUM
IF PRODNUM LT 'L99999'
END

If that runs successfully, we would continue our testing by moving the range of values, perhaps trying the same basic request, with

IF PRODNUM LT 'N99999'

Our goal is to find the range of values that cause the request to fail. Ultimately, we will end up with 2 tests, a GREATER THAN and a LESS THAN test. For our example, let's say when we run the following test, we receive the message SY0020: Incorrect linking address.

TABLEF
FILE PRODINFO
PRINT PRODNUM
IF PRODNUM LT 'U99999'
END

Based on the output from our earlier tests, this tells us the bad data exists in a record with a PRODNUM that is greater than the character string 'N99999', but less than the string 'U99999'. We can continue trying to isolate the data from both directions by testing for a value with both GREATER THAN and LESS THAN tests. In our sample case, let's say we are able to successfully run the following 2 requests:

TABLEF 				TABLEF
FILE PRODINFO			FILE PRODINFO
PRINT PRODNUM 			PRINT PRODNUM
IF PRODNUM LT 'S99999'		IF PRODNUM GT 'TB1111'
END				END

This has narrowed the data we will be eliminating to ONLY those records with a PRODNUM between S99999 and TB1111. Now that we've identified the corrupted data, what do we do about it? Well, there is nothing we can do to correct data once it has been corrupted. However, we can rebuild the database WITHOUT the bad data, thus minimizing the loss.

The following steps should be followed in order, and done precisely as described here. Before beginning, you will need to define three RAMSAVBx files and two separate RAMBACKU files, according to your operating system requirements. We want to create separate backups: one at the very beginning, which will contain the database with the corrupted data, and one in step 5, after the file with the bad data has been erased. The RAMSAVBx files may use any three unique suffix characters. For this example we will use F, 1 and 2.

For additional information on how to define these files to the host operating system, please refer to the CA Ramis publication Datasets.

Step 1)Create a CA Ramis BACKUP of the bad database.

BACKUP
Step 2)RDUMP the file description for the damaged file (PRODINFO) from RAMASTER using the RDUMPF command. Notice we are using the suffix "F" on the RDUMP command. Since we will be generating three RAMSAVBx files using three RDUMPxcommands, it is important to remember to use the suffix. This will prevent the multiple RDUMPx activities from over-writing the same dataset.

RDUMPF
FILE RAMASTER
IF FILENAME IS PRODINFO
END

Step 3)Perform 2 RDUMPx commands to capture as much data as possible from the damaged PRODINFO file. Again, we are using a suffix, in this case 1 and 2,to ensure separate RAMSAVBx files are being created.
RDUMP1				RDUMP2
FILE PRODINFO			FILE PRODINFO
IF PRODNUM LT 'S99999'		IF PRODNUM GT 'TB1111'
END				END
Step 4)Use the ERASE * command to remove the file definition and data for the PRODINFO file. Although not required, it is usually a good idea to run a RAMCLEAN utility after deleting data from the database.

ERASE*
FILE PRODINFO
END
RAMCLEAN

Step 5)Using the BACKUP utility, create a second backup copy of the damaged database. This copy will NOT have the bad data included. (Remember this backup should not overwrite the backup taken in Step 1.)

BACKUP

Step 6)Create a new version of the database by issuing the RELOAD command, to reload the backup created in Step 5.

USE 1
NEWDB
END
RELOAD

Step 7)Add the file description for PRODINFO into the newly created database:

REVISE
   INCLUDE FILENAME
   JUSTIFIED
   BINARY
   NATURAL FORM
   SEARCH FORWARD
   READ RAMSAVBF
FILE RAMASTER
END

(NOTE: This activity, and the 2 activities in Step 8, may be done using either Records Management or DMF. This example uses Records Management.)
Step 8)Load the data into file PRODINFO. This will be done as 2 steps, since we have 2 separate RAMSAVB files created from Step 3, RAMSAVB1 and RAMSAVB2.
REVISE				REVISE
INCLUDE FILENAME		 INCLUDE FILENAME
JUSTIFIED			 JUSTIFIED
BINARY				 BINARY
NATURAL FORM			 NATURAL FORM
SEARCH FORWARD		 SEARCH FORWARD
READ RAMSAVB1			 READ RAMSAVB2
FILE PRODINFO			FILE PRODINFO
END				END
Step 9)Run the following TABLE request to ensure the file description and data has loaded successfully into the "new" PRODINFO file.

TABLEF
FILE PRODINFO
PRINT PLANQTY
BY PRODNUM
END

That's all there is to it. Once you have verified the data in Step 9, you should create another final backup of the newly recreated database. As we mentioned earlier, the very best solution is to be sure to create backups frequently so if a problem does occur, you have a recent backup available.

Additional Information:
For more information on using the BACKUP and RELOAD commands, refer to the publication CA Ramis Utilities.
For information on using Records Management commands, refer to the publication CA Ramis Records Management.
For information on using DMF, refer to the publication CA Ramis Data Maintenance Facility.

All CA Ramis and CA Ramis Report manuals can be found in docops.ca.com 
https://docops.ca.com/bookshelves/ca-ramis
https://docops.ca.com/bookshelves/ca-ramis-reporter