SMF Records/How to Extract Values from SMF Record Fields

Summary
Processing the SMF record data with a REXX program in a batch environment consists roughly of the following steps:


 * Read the original SMF dataset (DUMPIN DD) with the IBM IFASMFDP program (see L11 or L12 )
 * Select the SMF record data you would like to process with a corresponding SYSIN DD Statement (see L11 or L12 for parameters)
 * Write formatted output into a new temporary dataset (DUMPOUT DD)
 * Convert the DUMPOUT dataset with the IBM IDCAMS REPRO Command into a readable format for REXX using a work dataset (WORKSMF DD)
 * Read all records from the WORKSMF dataset into a stem. variable in the memory of the REXX program
 * Process the required data and put the resulting comma-separated values file into another stem. variable for output
 * After finishing processing write this stem. variable into an output dataset (OUTDS DD) to store the resulting report

The Job Control Language
To process SMF record data with a REXX program in a batch environment there are a couple of prerequisites. First of all a formatted dump from the SMF dataset is needed, which can be produced with the IBM IFASMFDP program (see L11 or L12 ). Be sure to have at least read access to the SMF datasets in your installation. Unfortunately the resulting dataset from IFASMFDP has a RECFM(VBS) and is not readable directly in REXX. It has to be converted into another work dataset with RECFM(VB) and a suitable LRECL and BLKSIZE. This is done with the IBM IDCAMS REPRO command. Both the IFASMFDP and the REPRO commands are called from within the REXX script while the necessary DD statements are in the JCL itself.

//JOBNAME JOB 'ACCNR',.,CLASS=L,MSGCLASS=J, //         NOTIFY=&amp;SYSUID,MSGLEVEL=(1,1),LINES=200000 //* //* //* ==> SET SMFOUT TO SMF WORK DATASET NAME //* ==> SET REPOUT TO REPORT DATASET NAME //* //SET1   SET  SMFOUT=HLQ.XXX.READSMF.WRK //SET2   SET  REPOUT=HLQ.XXX.SMF19.DASDREP.SYSTEM //* //* ==> DELETE WORK AND REPORT DATASETS //* //KILL01  EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* DELETE HLQ.XXX.READSMF.WRK DELETE HLQ.XXX.SMF19.DASDREP.SYSTEM SET MAXCC=0 /* //* //* ==> START ISPF AND EXECUTE THE REXX //* //READSMF  EXEC  PGM=IKJEFT01,REGION=0M //* //ISPPROF DD UNIT=SYSDA,SPACE=(TRK,(9,1,4)), // DCB=(LRECL=80,BLKSIZE=3120,RECFM=FB,DSORG=PO), // DISP=(NEW,PASS) //* //* //ISPMLIB DD DISP=SHR,DSN=SYS1.ISP.SISPMENU //        DD DISP=SHR,DSN=SYS1.ISF.SISFMLIB //ISPPLIB DD DISP=SHR,DSN=SYS1.ISP.SISPPENU //        DD DISP=SHR,DSN=SYS1.ISF.SISFPLIB //ISPSLIB DD DISP=SHR,DSN=SYS1.ISP.SISPSENU //        DD DISP=SHR,DSN=SYS1.ISF.SISFSLIB //ISPTLIB DD DISP=SHR,DSN=SYS1.ISP.SISPTENU //        DD DISP=SHR,DSN=SYS1.ISF.SISFTLIB //ISPLOG  DD DUMMY //* --- //* ==> DUMPIN: SMF DATASET //* ==> SYSIN: RECORD TYPE, DAY AND TIME SELECTION //* --- //DUMPIN   DD    DISP=SHR,DSN=HLQ.SYSTEM.SMFDS.DJJMMDD //DUMPOUT  DD    DISP=(,PASS),SPACE=(CYL,(800,200)),UNIT=SYSDA, //         DSN=&amp;&amp;DP //SYSIN    DD    * INDD(DUMPIN,OPTIONS(DUMP)) OUTDD(DUMPOUT,TYPE(19)) DATE(2013334,2013334) START(0000) END(0100) /* //* //SYSEXEC  DD    DISP=SHR,DSN=HLQ.XXX.ISPF.EXEC //WORKSMF  DD    DSN=&amp;SMFOUT, // DCB=(LRECL=32756,BLKSIZE=32760,RECFM=VB,DSORG=PS), // DISP=(NEW,CATLG), // UNIT=USEPLA, // SPACE=(CYL,(100,10),RLSE) //OUTDS    DD    DSN=&amp;REPOUT, // DCB=(LRECL=800,BLKSIZE=31200,RECFM=FB,DSORG=PS), // DISP=(NEW,CATLG), // UNIT=USEPLA, // SPACE=(CYL,(10,10),RLSE) //SYSTSPRT DD    SYSOUT=* //SYSPRINT DD    SYSOUT=* //* //SYSTSIN  DD    * ISPSTART CMD(SMF19) /*

Chart 1: JCL for Data Selection and REXX Execution

In the example provided the first step KILL01 just deletes the work dataset and the resulting report dataset, because they are defined in the READSMF step as DISP=(NEW,CATLG).

In the READSMF step the program IKJEFT01 is executed to provide the necessary TSO/ISPF environment for calling the REXX program in batch.


 * All ISP* DD Statements belong to this environment.
 * The DD statements DUMPIN, DUMPOUT, SYSPRINT and SYSIN belong to the program IFASMFDP called at the beginning of the process_smf_data procedure in the REXX script.
 * In particular the SYSIN DD statement provides the possibility of shrinking the data to only the really necessary amount for processing. In this example data is limited to only the SMF19 Records of day 334 in 2013 within the timeframe 00:00 AM to 01:00 AM. To avoid too long execution times and errors because of redundant data or insufficient storage, it is strongly recommended to reduce the amount of data processed at this point to the data really needed.
 * The DUMPOUT DD statement defines only a temporary dataset with DSN=&amp;&amp;DP, because as mentioned before it is only needed for convertion into a work dataset with the REPRO Command called from within the REXX Script with "REPRO IFILE(DUMPOUT) OFILE(WORKSMF)".
 * The SYSPRINT DD statement is used to show the number of read and written SMF records by the program IFASMFDP in the job output


 * The SYSEXEC DD statement defines the dataset which contains the REXX script.
 * The WORKSMF DD statement defines the source dataset for the REXX processing. All SMF records contained in this dataset are put into the stem variable SMF.. SMF. is then processed by the REXX code and the resulting values are written into another stem variable DASD_OUT. in comma-separated values format. In fact here they are actually semicolon-separated values because of the spreadsheet number format in some countries.
 * The OUTDS DD statement defines the output dataset where the stem variable DASD_OUT. is written at the end of the REXX processing with the command "Execio * Diskw OUTDS (finis stem dasd_out."
 * The SYSTSPRT DD statement defines the part in the job output where the REXX script can write out messages i.e. with the say command. This is extremely useful for both documentation and debugging purposes.
 * The SYSTSIN DD statement contains the ISPSTART CMD(SMF19) command to start the REXX script with the member name SMF19

The REXX Script
The REXX scripts in this book are all more or less built on the same structure:


 * Part 1 consists of a initialize procedure which only writes a border line into the job output and a process_smf_data procedure which is the main program of the REXX
 * Part 2 consists of a function to compute the field offsets relative to the beginning of the record and some functions to convert different data formats into processable formats for output.

Because the function part (Part 2) is the same in all following examples it is only shown once in this complete example. It is not necessary to repeat it in all following examples. Nevertheless the universal function part is necessary and should be copied at the end of each following REXX example where indicated.

Chart 2: REXX SMF19 to create a .csv file with the values of interest

Let's now go through the main procedure process_smf_data.


 * As described in the JCL part the program IFASMFDP is called to get a formatted dump and the REPRO command is used to convert the dump output into the correct dataset format.
 * "EXECIO * DISKR WORKSMF ( FINIS STEM SMF." read all records of the working dataset into memory
 * The drop DASD_OUT. just clears the output variable DASD_OUT.
 * The counter variables count and j are then initialized to a value of 1 and a header with date, time and the number of read SMF records is written into the job output (SYSTSPRT).
 * After this initial work processing a conditional loop for each record begins

At this point you should check the description of the SMF19 record mapping(see L11 or L12 ) to track the different offsets and length of the used record fields. In most cases we used the field names from this description.

"Cyls_total = SMF19TRK / 15" "GB_total = (Cyls_total * 849960) / (1024 * 1024 * 1024)" GB_free = ((SMF19SPC * 15) + SMF19SPT) * (GB_total / SMF19TRK)
 * Now for each record the record header and type is extracted and a line is written to the job output containing the line number and the record type
 * If the record type is 19 some directly usable values such as Time, Date, SYSID, Volser, UCB Number, the unallocated cylinders and tracks and the total tracks on the volume can be extracted using appropriate conversion functions
 * From the total tracks on the volume one can derive the total cylinders used with the following formula:
 * From the size of SMF19TRK it can also be concluded to which DASD Model the volume belongs. The variable dasd_model is filled in the appropriate select statement.
 * Knowing that a 3390 device contains 849960 bytes per cylinder the total gigabytes of the volume can be obtained with the formula:
 * The following translate function in the REXX is used to mask the decimal point with a comma. REXX needs to compute with decimal points but the output dataset should contain a comma instead because of the spreadsheet number format in many countries.
 * The GB free and GB allocated of the volume can be derived from these formulas:

GB_alloc = GB_total - GB_free
 * Here we finish the record processing, increment the counter j by 1, write all native and derived values into the stem. variable dasd_out.j and increment the counter count by 1.
 * After processing the last record the value of count is written to dasd_out.0 and the report header line is written to dasd_out.1.
 * With "Execio * Diskw OUTDS (finis stem dasd_out." the whole report is then written to the output dataset defined for OUTDS.