Spool
This chapter describes spool data set and spool back-up mechanism.
1. Overview
The term 'spool' has multiple meanings; it can refer to the spool data set that accepts inputs to perform jobs, or it can refer to the volume where the spool data set that contains job execution results is located. It can also refer to an operation that uses the spool data set.
When a JCL file is submitted, TJES assigns a unique JOBID to the job and registers it in the JOBQ. Spool space for the job is allocated in the spool volume. Next, a spool data set named INPJCL is created, and the JCL is copied to this data set. INPJCL is used as the input for the rest of the job processing workflow.
In the spool space created previously, tjclrun then creates spool data sets named JESMSG, JESJCL, SYSMSG, and CATPROC, as well as some output spool data sets for the SYSOUT DD statements in the JCL.
Spool data sets can be inspected by using tjesmgr or OpenFrame Manager.
2. Spool Data Sets
The spool data set can be created in two different ways: default data sets (INPJCL, SYSMSG, CATPROC, CONVJCL, JESMSG, JESJCL) created by default during job execution, and output data sets created from JCL SYSOUT statements.
The following JCL code example copies content from TEST.DATASET1 to TEST.DATASET2 and TEST.DATASET3, and then deletes TEST.DATASET1.
<TESTPROC JCL>
//TESTPROC JOB CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1) //STEP01 EXEC IBGPROC //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,KEEP) //PS010.SYSUT2 DD DSN=TEST.DATASET2,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100) //STEP02 EXEC IBGPROC //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,DELETE) //PS010.SYSUT2 DD DSN=TEST.DATASET3,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100)
The following JCL code contains two steps, which use the CATALOGED PROCEDURE and IBGPROC commands. The content of IBGPROC and the use of the IEBGENER program are shown in the following.
<IBGPROC>
//PS010 EXEC PGM=IEBGENER //STEPLIB DD DSN=SYS1.USERLIB,DISP=SHR // DD DSN=SYS1.TEMPLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUT1 DD DUMMY //SYSUT2 DD DUMMY
After JCL TESTPROC is submitted, the following spool data sets are created.
SPOOL LIST : --------------------------------------------------------------------------- NO STEP DDNAME SIZE DSNAME --------------------------------------------------------------------------- 0 -------- INPJCL 409 INPJCL 1 -------- SYSMSG 5K SYSMSG 2 -------- CATPROC 444 CATPROC 3 -------- CONVJCL 791 CONVJCL 4 -------- JESMSG 1K JESMSG 5 -------- JESJCL 3K JESJCL 6 STEP01/PS010 SYSPRINT 463 oframe.TESTPROC.JOB00040.D000004 7 STEP02/PS010 SYSPRINT 463 oframe.TESTPROC.JOB00040.D000009 -----------------------------------------------------------------------------
2.1. Default Data Sets
The following describes default data sets.
INPJCL
Copies submitted JCL to the INPJCL spool data set, which is used for subsequent job processing.
//TESTPROC JOB CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1) //STEP01 EXEC IBGPROC //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,KEEP) //PS010.SYSUT2 DD DSN=TEST.DATASET2,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100) //STEP02 EXEC IBGPROC //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,DELETE) //PS010.SYSUT2 DD DSN=TEST.DATASET3,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100)
SYSMSG
Saves messages generated by tjclrun during job execution.
------------------------------------------------------------------------------ ** Cataloged Procedure IBGPROC in SYS1.PROCLIB SCAN start ** ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ ** Cataloged Procedure IBGPROC in SYS1.PROCLIB SCAN start ** ------------------------------------------------------------------------------ JOBID=JOB00040 JOBPOS=0 RUNPID=2454 TESTPROC JOB ============================================================================== --------------------------- STEP01 EXEC PROC step ---------------------------- (JRN0210I) Start parsing Cataloged Procedure ... ---------------------------- PS010 EXEC PGM step ----------------------------- (JRN0320I) SYSUT1 DD ALLOC - DSNAME=TEST.DATASET1 (JRN0320I) SYSUT2 DD ALLOC - DSNAME=TEST.DATASET2 (JRN0320I) STEPLIB DD ALLOC - DSNAME=SYS1.USERLIB (JRN0320I) CONCATED DD ALLOC - DSNAME=SYS1.TEMPLIB (JRN0321I) SYSPRINT DD ALLOC - DSNAME=JOB00040(oframe.TESTPROC.JOB00040.D000004) --------------------------- STEP02 EXEC PROC step ---------------------------- (JRN0210I) Start parsing Cataloged Procedure ... ---------------------------- PS010 EXEC PGM step ----------------------------- (JRN0320I) SYSUT1 DD ALLOC - DSNAME=TEST.DATASET1 (JRN0320I) SYSUT2 DD ALLOC - DSNAME=TEST.DATASET3 (JRN0320I) STEPLIB DD ALLOC - DSNAME=SYS1.USERLIB (JRN0320I) CONCATED DD ALLOC - DSNAME=SYS1.TEMPLIB (JRN0321I) SYSPRINT DD ALLOC - DSNAME=JOB00040(oframe.TESTPROC.JOB00040.D000009) (JRN0210I) Start parsing Cataloged Procedure ... (JRN0210I) Start parsing Cataloged Procedure ... ============================================================================== --------------------------- STEP01 EXEC PROC step ---------------------------- (JRN0210I) Start parsing Cataloged Procedure ... ---------------------------- PS010 EXEC PGM step ----------------------------- EXEC PGM=IEBGENER (JRN0129I) TACF CHECK UTILITY PROGRAM AUTH OK (JRN0320I) SYSUT1 DD ALLOC - DSNAME=TEST.DATASET1 (JRN0320I) SYSUT2 DD ALLOC - DSNAME=TEST.DATASET2 (JRN0320I) STEPLIB DD ALLOC - DSNAME=SYS1.USERLIB (JRN0320I) CONCATED DD ALLOC - DSNAME=SYS1.TEMPLIB (JRN0321I) SYSPRINT DD ALLOC - DSNAME=JOB00040(oframe.TESTPROC.JOB00040.D000004) (JRN0044I) dsalc_report_start OK (JRN0240I) SYSPRINT DD in STEP01/PS010 open (JRN0335I) SYSUT1 DD in STEP01/PS010 exported (JRN0335I) SYSUT2 DD in STEP01/PS010 exported (JRN0335I) STEPLIB DD in STEP01/PS010 exported (JRN0335I) SYSPRINT DD in STEP01/PS010 exported (JRN0056I) coprocess child: argv[0]=IEBGENER (JRN0058I) pgm pid=2460 (JRN0060I) Entering exec_pipe_loop() ... (JRN0301I) SYSIN DD to PGM done (JRN0245I) PGM closed stdout pipe (JRN0061I) Leaving exec_pipe_loop() ... (JRN0063I) pipe work done with coprocess child (PGM) (JRN0080I) PGM pid[2460] EXITED (JRN0065I) PS010 EXEC PGM step done with RC=0 (JRN0073I) dsalc_report_info OK (JRN0069I) step res usage: real[ 0.47] user[ 0.05] system[ 0.03] cuser[ 0.01] csystem[ 0.01] (JRN0070I) res usage: cpu_time[ 0.10] proc_time=[ 0.47] (JRN0345I) SYSUT1 DD DISPOSE COND=NORMAL (JRN0345I) SYSUT2 DD DISPOSE COND=NORMAL (JRN0345I) STEPLIB DD DISPOSE COND=NORMAL (JRN0345I) CONCATED DD DISPOSE COND=NORMAL (JRN0345I) SYSPRINT DD DISPOSE COND=NORMAL --------------------------- STEP02 EXEC PROC step ---------------------------- (JRN0210I) Start parsing Cataloged Procedure ... ---------------------------- PS010 EXEC PGM step ----------------------------- EXEC PGM=IEBGENER (JRN0129I) TACF CHECK UTILITY PROGRAM AUTH OK (JRN0320I) SYSUT1 DD ALLOC - DSNAME=TEST.DATASET1 (JRN0320I) SYSUT2 DD ALLOC - DSNAME=TEST.DATASET3 (JRN0320I) STEPLIB DD ALLOC - DSNAME=SYS1.USERLIB (JRN0320I) CONCATED DD ALLOC - DSNAME=SYS1.TEMPLIB (JRN0321I) SYSPRINT DD ALLOC - DSNAME=JOB00040(oframe.TESTPROC.JOB00040.D000009) (JRN0044I) dsalc_report_start OK (JRN0240I) SYSPRINT DD in STEP02/PS010 open (JRN0335I) SYSUT1 DD in STEP02/PS010 exported (JRN0335I) SYSUT2 DD in STEP02/PS010 exported (JRN0335I) STEPLIB DD in STEP02/PS010 exported (JRN0335I) SYSPRINT DD in STEP02/PS010 exported (JRN0056I) coprocess child: argv[0]=IEBGENER (JRN0058I) pgm pid=2467 (JRN0060I) Entering exec_pipe_loop() ... (JRN0301I) SYSIN DD to PGM done (JRN0245I) PGM closed stdout pipe (JRN0061I) Leaving exec_pipe_loop() ... (JRN0063I) pipe work done with coprocess child (PGM) (JRN0080I) PGM pid[2467] EXITED (JRN0065I) PS010 EXEC PGM step done with RC=0 (JRN0073I) dsalc_report_info OK (JRN0069I) step res usage: real[ 0.42] user[ 0.04] system[ 0.02] cuser[ 0.02] csystem[ 0.02] (JRN0070I) res usage: cpu_time[ 0.10] proc_time=[ 0.42] (JRN0345I) SYSUT1 DD DISPOSE COND=NORMAL (JRN0345I) SYSUT2 DD DISPOSE COND=NORMAL (JRN0345I) STEPLIB DD DISPOSE COND=NORMAL (JRN0345I) CONCATED DD DISPOSE COND=NORMAL (JRN0345I) SYSPRINT DD DISPOSE COND=NORMAL --------------------------- DATA SET UNALLOCATION ---------------------------- (JRN0346I) SYSPRINT DD UNALLOC COND=NORMAL (JRN0346I) STEPLIB DD UNALLOC COND=NORMAL (JRN0346I) SYSUT2 DD UNALLOC COND=NORMAL (JRN0346I) SYSUT1 DD UNALLOC COND=NORMAL (JRN0346I) SYSPRINT DD UNALLOC COND=NORMAL (JRN0346I) STEPLIB DD UNALLOC COND=NORMAL (JRN0346I) SYSUT2 DD UNALLOC COND=NORMAL (JRN0346I) SYSUT1 DD UNALLOC COND=NORMAL (JRN0333I) JOB is enqueued for output processing - JOBID=JOB00040, ABEND=0
The system resources, which are recorded in step res usage or res usage and used during the step, are as follows.
-
step res usage
Field Description real
Execution time of the step (in seconds).
user
Time that the step uses CPU in the user area (in seconds).
system
Time that the step uses CPU in the system area (kernel mode) (in seconds).
cuser
Time that the child process created in the step uses CPU in the user area (in seconds).
csystem
Time that the child process created in the step uses CPU in the system area (kernel mode) (in seconds).
-
res usage
Field Description cpu_time
Time that the step uses CPU (in seconds).
proc_time
Same as "real" in step res usage. Execution time of the step (in seconds).
memory size
Fixed to -1.
CATPROC
Stores contents of catalogued procedure used during job execution.
XX-------- IBGPROC in SYS1.PROCLIB --------XX //PS010 EXEC PGM=IEBGENER //STEPLIB DD DSN=SYS1.USERLIB,DISP=SHR // DD DSN=SYS1.TEMPLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUT1 DD DUMMY //SYSUT2 DD DUMMY XX-------- IBGPROC in SYS1.PROCLIB --------XX //PS010 EXEC PGM=IEBGENER //STEPLIB DD DSN=SYS1.USERLIB,DISP=SHR // DD DSN=SYS1.TEMPLIB,DISP=SHR //SYSPRINT DD SYSOUT=* //SYSUT1 DD DUMMY //SYSUT2 DD DUMMY
CONVJCL
Stores replaced procedures and parameter values deployed during job execution. Catalogued procedures are printed as 'XX' and JCL instream procedure, '++.'
//TESTPROC JOB CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1) //STEP01 EXEC IBGPROC XX$DUMMY$ PROC XXPS010 EXEC PGM=IEBGENER XXSTEPLIB DD DSN=SYS1.USERLIB,DISP=SHR XX DD DSN=SYS1.TEMPLIB,DISP=SHR XXSYSPRINT DD SYSOUT=* //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,KEEP) X/SYSUT1 DD DUMMY //PS010.SYSUT2 DD DSN=TEST.DATASET2,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100) X/SYSUT2 DD DUMMY //STEP02 EXEC IBGPROC XX$DUMMY$ PROC XXPS010 EXEC PGM=IEBGENER XXSTEPLIB DD DSN=SYS1.USERLIB,DISP=SHR XX DD DSN=SYS1.TEMPLIB,DISP=SHR XXSYSPRINT DD SYSOUT=* //PS010.SYSUT1 DD DSN=TEST.DATASET1,DISP=(SHR,DELETE) X/SYSUT1 DD DUMMY //PS010.SYSUT2 DD DSN=TEST.DATASET3,DISP=(NEW,CATLG), // DCB=(RECFM=FB,LRECL=10,BLKSIZE=100) X/SYSUT2 DD DUMMY
|
The entire content is printed on CONVJCL only when the JCL execution is fully successful. If a syntax error occurs during the execution, CONVJCL is created with content partially printed or not printed at all. |
JESMSG
Stores information about a job and each specific step of the job.
---< JOB INFO >-----------------------------------------------------------------
JOB ID : JOB00040
JOB NAME : TESTPROC NODENAME : NODE1
JOB CLASS : A , JOB STATUS : D(R00000) , JOB PRTY : 5 , RUNNER INDEX : 8
JOB USER : ROOT
JCL PATH : /home/oframe/OpenFrame/volume_default/SYS1.JCLLIB/TESTPROC
TIME STAMP : SUBM 20200122164142, EXEC 20200122164143, TERM 20200122164145
RES USAGE : CPU - 0s (p:1s) , 0%
---<STEP INFO>------------------------------------------------------------------
STEP : [STEP01/PS010]
CPU 0s(p: 0s)
SYSUT1 TEST.DATASET1 R:10 W:0
SYSUT2 TEST.DATASET2 R:0 W:10
STEPLIB SYS1.USERLIB R:0 W:0
SYS1.TEMPLIB R:0 W:0
SYSPRINT JOB00040(oframe.TESTPROC.JOB00040.D000004) R:0 W:5
STEP : [STEP02/PS010]
CPU 0s(p: 0s)
SYSUT1 TEST.DATASET1 R:10 W:0
SYSUT2 TEST.DATASET3 R:0 W:10
STEPLIB SYS1.USERLIB R:0 W:0
SYS1.TEMPLIB R:0 W:0
SYSPRINT JOB00040(oframe.TESTPROC.JOB00040.D000009) R:0 W:5
The following describes the information about jobs and steps stored in JESMSG.
-
Job Information
Field Description JOB ID
Unique number of the allocated job.
JOB NAME
Name of the job.
NODENAME
Name of the TJES node in which the job is executed.
JOB CLASS
Job class.
JOB STATUS
Current status of the job (returned value).
JOB PRTY
Priority of scheduling of the job.
RUNNER INDEX
Number of Runner slot that executes the job.
JOB USER
User of the job.
JCL PATH
Path and name of the JCL file submitted by the user.
TIME STAMP
Time of job submission, and start/end time of job execution.
RES USAGE
CPU time used for job execution.
-
CPU
-
0s: Total CPU usage of tjclrun and forked child processes. (Since the current process has ended, the value is 0.)
-
(p:2s): Elapsed time between tjclrun forking starts and ends (job execution time)
-
0%: CPU usage in relation to job execution time
-
-
Step Information
Field Description STEP
Name of the job step.
CPU
CPU time used for the job step (actual processing time).
SYSUT1
SYSUT1 I/O count from the job step.
SYSUT2
SYSUT2 I/O count from the job step.
STEPLIB
STEPLIB I/O count from the job step.
SYSPRINT
SYSPRINT I/O count from the job step.
JESJCL
Stores the parse tree for INPJCL’s JCL statement.
JOB STREAM=[/home/oframe/OpenFrame/spool/JOB00040/INPJCL], JOBPOS=[0]
========================================================================
/JOB[name=TESTPROC,lineno=1]={
|[key=CLASS]={STR,value=[A]}
|[key=MSGCLASS]={STR,value=[X]}
|[key=MSGLEVEL]={SUB,value={
| |[pos=1]={STR,value=[1]}
| |[pos=2]={STR,value=[1]}
|}}
|-------- child statements --------
|/EXEC[name=STEP01,lineno=2]={
| |[pos=1]={STR,value=[IBGPROC]}
|}
|/DD[name=PS010.SYSUT1,lineno=3]={
| |[key=DSN]={STR,value=[TEST.DATASET1]}
| |[key=DISP]={SUB,value={
| | |[pos=1]={STR,value=[SHR]}
| | |[pos=2]={STR,value=[KEEP]}
| |}}
|}
|/DD[name=PS010.SYSUT2,lineno=4]={
| |[key=DSN]={STR,value=[TEST.DATASET2]}
| |[key=DISP]={SUB,value={
| | |[pos=1]={STR,value=[NEW]}
| | |[pos=2]={STR,value=[CATLG]}
| |}}
| |[key=DCB]={SUB,value={
| | |[key=RECFM]={STR,value=[FB]}
| | |[key=LRECL]={STR,value=[10]}
| | |[key=BLKSIZE]={STR,value=[100]}
| |}}
|}
|/EXEC[name=STEP02,lineno=6]={
| |[pos=1]={STR,value=[IBGPROC]}
|}
|/DD[name=PS010.SYSUT1,lineno=7]={
| |[key=DSN]={STR,value=[TEST.DATASET1]}
| |[key=DISP]={SUB,value={
| | |[pos=1]={STR,value=[SHR]}
| | |[pos=2]={STR,value=[DELETE]}
| |}}
|}
|/DD[name=PS010.SYSUT2,lineno=8]={
| |[key=DSN]={STR,value=[TEST.DATASET3]}
| |[key=DISP]={SUB,value={
| | |[pos=1]={STR,value=[NEW]}
| | |[pos=2]={STR,value=[CATLG]}
| |}}
| |[key=DCB]={SUB,value={
| | |[key=RECFM]={STR,value=[FB]}
| | |[key=LRECL]={STR,value=[10]}
| | |[key=BLKSIZE]={STR,value=[100]}
| |}}
|}
}
========================================================================
CATALOGED PROCEDURE=[IBGPROC]
========================================================================
/PROC[name=$DUMMY$,lineno=2]={
|-------- child statements --------
|/EXEC[name=PS010,lineno=3]={
| |[pos=1,key=PGM]={STR,value=[IEBGENER]}
|}
|/DD[name=STEPLIB,lineno=4]={
| |[key=DSN]={STR,value=[SYS1.USERLIB]}
| |[key=DISP]={STR,value=[SHR]}
|}
|/DD[name=,lineno=5]={
| |[key=DSN]={STR,value=[SYS1.TEMPLIB]}
| |[key=DISP]={STR,value=[SHR]}
|}
|/DD[name=SYSPRINT,lineno=6]={
| |[key=SYSOUT]={STR,value=[*]}
|}
|/DD[name=SYSUT1,lineno=7]={
| |[pos=1]={STR,value=[DUMMY]}
|}
|/DD[name=SYSUT2,lineno=8]={
| |[pos=1]={STR,value=[DUMMY]}
|}
}
/PEND[name=,lineno=8]={
}
CATALOGED PROCEDURE=[IBGPROC]
========================================================================
/PROC[name=$DUMMY$,lineno=2]={
|-------- child statements --------
|/EXEC[name=PS010,lineno=3]={
| |[pos=1,key=PGM]={STR,value=[IEBGENER]}
|}
|/DD[name=STEPLIB,lineno=4]={
| |[key=DSN]={STR,value=[SYS1.USERLIB]}
| |[key=DISP]={STR,value=[SHR]}
|}
|/DD[name=,lineno=5]={
| |[key=DSN]={STR,value=[SYS1.TEMPLIB]}
| |[key=DISP]={STR,value=[SHR]}
|}
|/DD[name=SYSPRINT,lineno=6]={
| |[key=SYSOUT]={STR,value=[*]}
|}
|/DD[name=SYSUT1,lineno=7]={
| |[pos=1]={STR,value=[DUMMY]}
|}
|/DD[name=SYSUT2,lineno=8]={
| |[pos=1]={STR,value=[DUMMY]}
|}
}
/PEND[name=,lineno=8]={
}
2.2. Data Sets Defined by SYSOUT in JCL
These data sets receive output from programs that are called in the JCL file. In the following example, a spool data set is created for each SYSPRINT statement. These save the messages from STEP01 and STEP02 of the IEBGENER program.
<STEP01/PS010 SYSPRINT oframe.TESTPROC.JOB00040.D000004>
[2020-01-22T16:41:43.906976] [IEBGENER(2460) ] [M] [IBG0001M] =====< IEBGENER START >===== [2020-01-22T16:41:43.911809] [IEBGENER(2460) ] [M] [IBG0005M] RECORD COUNT = 10 [2020-01-22T16:41:43.911817] [IEBGENER(2460) ] [M] [IBG0006M] TOTAL RECORD COUNT = 10 [2020-01-22T16:41:43.934682] [IEBGENER(2460) ] [M] [IBG0002M] =====< IEBGENER FINISH >=====
<STEP02/PS010 SYSPRINT oframe.TESTPROC.JOB00040.D000009>
[2020-01-22T16:41:44.490414] [IEBGENER(2467) ] [M] [IBG0001M] =====< IEBGENER START >===== [2020-01-22T16:41:44.493186] [IEBGENER(2467) ] [M] [IBG0005M] RECORD COUNT = 10 [2020-01-22T16:41:44.493194] [IEBGENER(2467) ] [M] [IBG0006M] TOTAL RECORD COUNT = 10 [2020-01-22T16:41:44.509200] [IEBGENER(2467) ] [M] [IBG0002M] =====< IEBGENER FINISH >=====
3. Spool Backup
When an unnecessary job is removed from the JOBQ, the associated spool is removed as well. If the spool data needs to be preserved, the spool backup feature can be used to create spool backups.
When a spool backup command is issued, a directory with the current date is created in the backup directory specified to the VALUE of the SPOOL_BACKUP_DIR key in the SPOOL section of the tjes subject, under OpenFrame Configuration.
After that, a new file named JOBID_JOBNAME_submitDATE_submitTIME.tar.gz is created in the new spool directory; this file is an archive of the target job’s spool. The backed-up job is then removed from JOBQ and its spool is deleted. To find the backup spool, untar and unzip the archive file to the temp directory. This directory is specified to the VALUE of the SPOOL_UNPACK_DIR key in the SPOOL section of the tjes subject, under OpenFrame Configuration.
Spool backup is available when the job is in DONE, ERROR, STOP, or FLUSH state.
|