Embed Size (px)
Citation preview
Divide and Conquer: Dealing with 140GB of SMF Data Daily
Chuck Hopf
Merrill Consultants
Gathering Data - Steps
Determining what data is available Determining what data is needed Determining data retention
requirements Gathering the data Dealing with increasing volumes
Data Availability
Different OS’s have different sources For MVS
– SMF – RMF– Vendor SMF records– Other software logs/sources
SMF/RMF
Job accounting System utilization CICS Transactions CICS Statistics DB2 Accounting DB2 Statistics Dataset activity
Vendor SMF Records
NETSPY Stop X37 MXG Tape Mount Monitor IND$FILE TPX HSM many many more
Other Data Sources
DCOLLECT - DASD space utilization CA1 - Tape utilization TMON - alternative CICS/DB2
transaction data source SYSLOG - console commands many more
Data Requirements
Not all data is always useful Largely driven by reporting requirements But… some can be discarded outright and
others cannot– type 4 5 34 35 are obsolete – other types are not generally useful– for our purposes, SMF types 4 5 16 19 34
35 40 and 69 are suppressed
Data Rentention
How long do you keep the raw data?– Ask your internal auditors – At least partially driven by volume
at 120GB/day, keeping CICS detail for very long is impractical and DB2 is not far behind
Third largest volume is type 74 RMF data Fourth is type 42
Gathering the Data
SMF records to a SYS1.xxxxxx dataset When the dataset is full, it is
automatically switched to a non-full dataset and (if the IEFU29 exit is coded) a dump/clear process is started
Gathering the Data
Best case - a single MAN dataset lasts all day– Dump and clear after a switch at ‘midnight’
Worst case - a MAN dataset fills every 15-20 minutes or less– Dump and clear frequently and consolidate
after ‘midnight’
Gathering the Data
‘Midnight’ may not be midnight. It may depend on a batch cycle and the availability of storage devices (tape drives)
The process is driven by the volume of data
Flies in the Ointment
Lost data can occur– When buffer expansions occurs– When MANx dataset fills during an interval
‘pop’– When something slows down the IO to the
MANx volume
Flies in the Ointment
CICS and DB2 may complain about SMF not being available
Can result in CICS slowdowns and response time problems
Flies in the Ointment
At 10MB/second, it takes about 270 seconds to fill a 3390-3
At 10MB/second, it takes about 600 seconds to dump and clear a volume
We are running at about 600-900 seconds at peak loads to fill a volume
What happens if volumes double? Can the SMF writer handle the load?
Fly Swatters
Must have at least two SYS1.Manx datasets and three is better– As volumes increase more may be needed
Make each SYS1.MANx dataset a full volume– Eliminates IOSQ delays– Eliminates device pending delays– Prevents problems
Fly Swatters
Add an auto operations rule:– 5 minutes before the intervals are due to
‘pop’ issue a D SMF command and if the active MAN volume is more than 85% full, issue an I SMF command.
– Eliminates filling the MAN volume during the interval processing in all but some extreme (and bizarre) cases
Fly Swatters
Use Maximum Values for Buffer sizes Tune CI Size on MAN datasets to
match your workload APARs are in progress to change the
buffer acquisition algorithms
Fly Swatters
Turn off CACHE in RMF on all but a single system. It will not only reduce the volume of data but will reduce the CPU time consumed by RMF. Do the same with RMF III (it is a BIG CPU reduction with RMF III.)
The Flies Will Win
Unless changes are made in the way the SMF writer works, we are going to break it in the near term (1-2 years?) It will simply not be able to keep up with the arrival rate of data.
Gathering the Data - Simple Case
MAN datasetIFASMFDP
Dump and ClearDaily SMF
Dealing with Volume
Treat it as you would any other application
Parallelism is the answer as volumes increase
Dealing with Volume
Split the data using IFASMFDP into chunks of a manageable size
No arcane exits just simple control statements– OUTDD(CICS,TYPE(110)) – OUTDD(DB2,TYPE(100:102))– OUTDD(SMFDUMP,NOTYPE(100:102,110))
Dealing with Volume
MAN DatasetIFASMFDP
Dump and Clear
Daily CMFData
Daily DB2Data
Daily SMFData
Less Simple Case
Dealing with Volume
As volumes grows, the structure can grow with it - up to a point
Dealing with Volume
MAN Dataset IFASMFDP
Other SMF
DB2 Data
CICS Data
IFASMFDP
IFASMFDP
DailyCICSData
DailyDB2Data
DailySMFData
Complex Case
IFASMFDP
Dealing with Volume
At some point, it gets too huge to process in a single piece.
Time to use the IFASMFDP exits Break CICS/DB2 up by
APPLID/SUBSYSTEM
Dealing with Volume
MANdataset
IFASMFDP
OtherSMF
DB2
CICS3
CICS2
CICS1
Dealing with Volume
SMF data can be piped with batch pipes– Must be VB NOT VBS and it must be set to
VB at the initial dump of the MAN volume– Best not to pipe when dumping MAN
volumes The dump runs at the speed of the downstream
process which may cause problems (dump/clear process running slower than next MAN volume fills.)
Piping SMF
If you intend to pipe SMF data, you would be wise to put in an IFASMFDP exit to catch any records larger than 32756 and either route them to a separate VBS file or discard them. There are not supposed to be any but there are (type 8 and type90.32 - APARs pending.)
Building the PDB
Before you start– Customizing– Retention– What cycles to run
Customizing
Define workloads Define shifts Define accounting fields Define user records to be added Define variables to be kept To compress or to not compress - that
is a question
Defining Workloads
Used to be limited to 15 workloads new limit is 114 (but that does not mean you should have 114 - more than 20 becomes cumbersome.)
Was restricted to control performance groups may now use report performance groups (be careful not to double dip.)
Defining Workloads
Two members affected– IMACWORK - old method– RMFINTRV - new method
Defining Workloads - IMACWORK
Simple IF THEN ELSE logic– IF PEFGRP=2 THEN WORK=‘TSO’;– IF SRVCLASS=‘TSO’ then WORK=‘TSO’;
Defining Workloads - RMFINTRV
Workloads defined using a parameter passed to a MACRO
Each workload (WORK1-WORK99) parameter has five components separated with a /– Name - 3-4 character description– 9 characters to be used in label describing workload– list of performance groups to include in the workload– list of service classes to include in the workload– number of periods in perfgrp/service class
Defining Workloads - RMFINTRV
Performance groups and service classes can be mixed
Report and control groups can be mixed– But… it is not a good idea. If you are going
to use report groups use NOTHING but report groups
Defining Workloads - RMFINTRV
%VMXGRMFI( …. WORK1=BATT/Test Jobs/1 3/BATCHLO, WORK2=TSPD/Dev TSO/2/DEVTSO/2, WORK3=DB2A/DB2A/ /DB2A, WORK4=DB2B/DB2B/5 …);
Defining Shifts
How are shifts important in your world?– Batch cycle vs online– Operations shifts– Weekends– Holidays
Defining Shifts - Holidays
Defining Shifts - Holidays
Holidays usually look like a weekend day Can distort weekly data and plans Should you or should you not exclude them Let the best technicians in your organization
(the managers) make the call
Defining Shifts - IMACSHFT
More simple IF THEN ELSE logic
IF 8 LE HOUR(TIMEPART(DATETIME)) LE 16 AND 2 LE WEEKDAY(DATEPART(DATETIME) LE 6 THEN DO: SHIFT=‘P’; DATETIME=DHMS(DATEPART(DATETIME),8,0,0);END;ELSE IF ...
Defining Shifts - $SHIFT
The first question when you present a report will be ‘What does this P mean under SHIFT?’ It will not matter how many other reports show the same thing or how many times you explain it.
A user format solves the problem.
Defining Shifts - $SHIFT
PROC FORMAT LIB=LIBRARY; VALUE $SHIFT ‘P’=‘First ’ ‘S’=‘Second’ ‘T’=‘Third ’ other=‘Weekend/Holiday’;
Defining Accounting Fields
How many do you really really need? More than 2 or 3 will start to be
redundant Limit the size and number
Defining Accounting Fields - IMACACCT
Assume 3 fields each 5 bytes long
DROP ACCOUNT4-ACCOUNT9 SACCT4-SACCT9 LENACCT4-LENACCT9;LENGTH ACCOUNT4-ACCOUNT9 SACCT1-SACCT9 $ 5;
Define User Records
Originally required multiple exits Now done ‘instream’ in BUILDPDB
process Still invokes the original exits
– EXPDBINC - include the source code– EXPDBVAR - build the datasets/variables– EXPDBCDE - read the data– EXPDBOUT - sort the data into the PDB
Define User Records
Each user record must have an SMF ID defined
These should be stored in IMACKEEP
MACRO _IDTPX 205 % /* TPX */MACRO _NSPYID 132 % /* NETSPY */MACRO _SYNCID 208 % /* SYNCSORT */etc...
Define User Records
Modify SYSIN
%LET EPDBINC=%QUOTE( VMACNSPY VMACTPX VMACSYNC ...);%LET EPDBVAR=%QUOTE( _VARNSPY _VARTPX _VARSYNC ...);
Define User Records
%LET EPDBCDE=%QUOTE( _CDESPY _CDETPX _CDESYNC ...);%LET EPDBOUT=%QUOTE( _SNSPY _STPX _SSYNC ...);
Defining Kept Variables
Can be done using the ‘MACKEEP’ macro variable to redefine the _Vdddddd macro for a dataset or by using the _Kdddddd macro for the dataset.
Defining Kept Variables
Using MACKEEP - drop ZDATE UNITADR UCBTYPE
%LET MACKEEP=%QUOTE(MACRO _VTY21 _WTY21 /* TYPE21 */ (LABEL='TY21: TYPE 21 SMF - TAPE ERROR STATS' KEEP=BLKSIZE BYTEREAD BYTEWRIT CLEAN DCBOFLG DENSITY DEVICE DEVNR ERASE ERRORS LCU NOISE OPEN PERMREAD PERMWRIT SIOCOUNT SMFTIME SYSTEM TAPCUSER TEMPREAD TEMPRBER TEMPRFER TEMPWRER TEMPWRIT VOLSER )%);
Defining Kept Variables
Using _Kdddddd - drop ZDATE UNITADR UCBTYPE
%LET MACKEEP=%QUOTE(MACRO _KTY21 DROP=ZDATE UNITADR UCBTYPE%);
Defining Kept Variables
Using MACKEEP - ADD variable DRIVE %LET MACKEEP=%QUOTE(MACRO _VTY21 _WTY21 /* TYPE21 */ (LABEL='TY21: TYPE 21 SMF - TAPE ERROR STATS' KEEP=BLKSIZE BYTEREAD BYTEWRIT CLEAN DCBOFLG DENSITY DEVICE DEVNR ERASE ERRORS LCU NOISE OPEN PERMREAD PERMWRIT SIOCOUNT SMFTIME SYSTEM TAPCUSER TEMPREAD TEMPRBER TEMPRFER TEMPWRER TEMPWRIT UCBTYPE UNITADR VOLSER ZDATE DRIVE )%);
Defining Kept Variables
Using _Kdddddd - add variable DRIVE
%LET MACKEEP=%QUOTE(MACRO _KTY21 DRIVE%);
Defining Kept Variables
Using MACKEEP - ADD variable DRIVE %LET MACKEEP=%QUOTE(MACRO _VTY21 _WTY21 /* TYPE21 */ (LABEL='TY21: TYPE 21 SMF - TAPE ERROR STATS' KEEP=BLKSIZE BYTEREAD BYTEWRIT CLEAN DCBOFLG DENSITY DEVICE DEVNR ERASE ERRORS LCU NOISE OPEN PERMREAD PERMWRIT SIOCOUNT SMFTIME SYSTEM TAPCUSER TEMPREAD TEMPRBER TEMPRFER TEMPWRER TEMPWRIT UNITADR VOLSER ZDATE DRIVE )%);
Defining Kept Variables
Using _Kdddddd - add variable DRIVE and drop variable UCBTYPE
%LET MACKEEP=%QUOTE(MACRO _KTY21 DRIVE DROP=UCBTYPE%);
Which Technique is Correct?
Both– First listing all variables makes it more
clear– Second is a lot less typing
Confused?
Use UTILBLDP– based on parameters you pass, constructs
the SYSIN for a tailored BUILDPDB
Reducing Confusion
Assume:– No CICS processing– No DB2 processing– No TYPE74 processing– Add SYNCSORT records as ID=200– Add TSO/MON records as ID=205/206
Reducing Confusion
%UTILBLDP( SUPPRESS=110 DB2 74, SPINCNT=7, USERADD=SYNC/200 TSOM/205, TIMEDIF=0, BUILDPDB=YES, OUTFILE=MYBUILD);
Reducing Confusion
/**********************************************************/ /* COPYRITE 1999 MERRILL CONSULTANTS DALLAS TX USA */ /* ARTIFICIALLY CONSTRUCTED SYSIN FOR MXG 18.18 */ /* SMF PROCESSING. BUILT USING UTILBLDP. */ %LET MACKEEP=%QUOTE( MACRO _SPINCNT 7 % /* SPIN COUNTER */ MACRO _SPINUOW 0 % /* UOW SPIN COUNTER */ MACRO _TIMEDIF 0 % /* TIME DIFFERENCE */ /* MXG STRONGLY RECOMMENDS PUTTING THE FOLLOWING*/ /* MACRO DEFINITIONS FOR THE ID MACROS IN YOUR */ /* USERID.SOURCLIB RATHER THAN INSTREAM. */ MACRO _SYNCID 200 % /* YOU MAY NEED TO CHANGE THESE MACROS*/ /* BECAUSE THERE IS MORE THAN A SINGLE*/ /* SMF TYPE FOR THESE RECORDS AND THE*/ /* UTILBLDP SYNTAX CAN ONLY HANDLE ONE.*/ MACRO _TSOMCMD 205 % MACRO _TSOMSYS 206 %
Reducing Confusion
MACRO _VARDB2 % /* SUPPRESS SMF RECORD TYPE(S)*/ MACRO _CDEDB2 IF 99 = -99 THEN RETURN;% MACRO _SDB2 % /* NO SORT */ MACRO _DIFFDB2 % /* NO DIFF CODE */ MACRO _VAR110 % /* SUPPRESS SMF RECORD TYPE(S)*/ MACRO _CDE110 IF 99 = -99 THEN RETURN;% MACRO _S110 % /* NO SORT */ MACRO _SCICEXC % /* CICS RECORDS BYPASSED */ MACRO _SCICSYS %
Reducing Confusion
MACRO _ETY74 % /* NO OUTPUT */ MACRO _ETY74CA % /* NO OUTPUT */ MACRO _ETY74CF % /* NO OUTPUT */ MACRO _ETY74CO % /* NO OUTPUT */ MACRO _ETY74LK % /* NO OUTPUT */ MACRO _ETY74ME % /* NO OUTPUT */ MACRO _ETY74OM % /* NO OUTPUT */ MACRO _ETY74PA % /* NO OUTPUT */ MACRO _ETY74ST % /* NO OUTPUT */ MACRO _ETY74SY % /* NO OUTPUT */ MACRO _ETY74TD % /* NO OUTPUT */ MACRO _ETY746B % /* NO OUTPUT */ MACRO _ETY746F % /* NO OUTPUT */ MACRO _ETY746G % /* NO OUTPUT */
Reducing Confusion
MACRO _LTY74 _WTY74 % /* STAY IN WORK */ MACRO _LTY74CA _WTY74CA % /* STAY IN WORK */ MACRO _LTY74CF _WTY74CF % /* STAY IN WORK */ MACRO _LTY74CO _WTY74CO % /* STAY IN WORK */ MACRO _LTY74LK _WTY74LK % /* STAY IN WORK */ MACRO _LTY74ME _WTY74ME % /* STAY IN WORK */ MACRO _LTY74OM _WTY74OM % /* STAY IN WORK */ MACRO _LTY74PA _WTY74PA % /* STAY IN WORK */ MACRO _LTY74ST _WTY74ST % /* STAY IN WORK */ MACRO _LTY74SY _WTY74SY % /* STAY IN WORK */ MACRO _LTY74TD _WTY74TD % /* STAY IN WORK */ MACRO _LTY746B _WTY746B % /* STAY IN WORK */ MACRO _LTY746F _WTY746F % /* STAY IN WORK */ MACRO _LTY746G _WTY746G % /* STAY IN WORK */ MACRO _S74 % /* NO SORT */
Reducing Confusion
/* WARNING: ONE OR MORE OF THE RMF RECORDS NEEDED */ /* WARNING: BY RMFINTRV HAS BEEN SUPPRESSED. SOME */ /* WARNING: FIELDS MAY BE EMPTY IN RMFINTRV. */ /* WARNING: SUPPRESSED RECORDS ARE: 74 */ MACRO _VARUSER /* USER SMF _VAR DEFINITIONS */ _VARSYNC _VARTSOM % MACRO _CDEUSER /* USER SMF _CDE DEFINITIONS */ _CDESYNC _CDETSOM % MACRO _EPDBOUT /* USER SMF OUTPUT DEFINITIONS */ _SSYNC _STSOM % );
Reducing Confusion
%LET PTY74=WORK; /* NO OUTPUT */ %LET PTY74CA=WORK; /* NO OUTPUT */ %LET PTY74CF=WORK; /* NO OUTPUT */ %LET PTY74CO=WORK; /* NO OUTPUT */ %LET PTY74LK=WORK; /* NO OUTPUT */ %LET PTY74ME=WORK; /* NO OUTPUT */ %LET PTY74OM=WORK; /* NO OUTPUT */ %LET PTY74PA=WORK; /* NO OUTPUT */ %LET PTY74ST=WORK; /* NO OUTPUT */ %LET PTY74SY=WORK; /* NO OUTPUT */ %LET PTY74TD=WORK; /* NO OUTPUT */ %LET PTY746B=WORK; /* NO OUTPUT */ %LET PTY746F=WORK; /* NO OUTPUT */ %LET PTY746G=WORK; /* NO OUTPUT */
Reducing Confusion
/* NOW RUN BUILDPDB */ %INCLUDE SOURCLIB(BUILDPDB); %INCLUDE SOURCLIB(ASUM70PR); /* RECOMMENDED */ %INCLUDE SOURCLIB(ASUMTAPE); /* RECOMMENDED */ %INCLUDE SOURCLIB(ASUMTMNT); /* RECOMMENDED */ %INCLUDE SOURCLIB(ASUMTALO); /* RECOMMENDED */
Compression?
Space can become a problem but compressing the data comes at a CPU cost of 2-3% minimum
Two options:– SAS compression – Hardware compression
SAS Compression
Applies only to DASD format SAS datasets
Invoked at the dataset level via a dataset option or at the system level via the COMPRESS=YES option
With version 6 of SAS, it may help you avoid the problems of multi-volume WORK datasets
Hardware Compression
Applies only to tape format SAS datasets on disk
May be striped May be compressed May be striped and compressed Only a single SAS dataset per DD can be
open at any point in time Useful for things like CICSTRAN DB2ACCT
Retaining Data
Weekly/Monthly Processing vs WTD/MTD processing
TREND processing How long?
Processing Cycle
BUILDPDB Monday? WEEKBLD1st
Month?MONTHBLD
YES
NO
YES
STOP
NO
DailySMF
7 DAILYPDBs
5Weekly
PDB
MonthlyPDB
Processing Cycle
Original Implementation Circa 1983 Worked when SMF volume was small Breaks down as volumes increase
without major changes– primarily a space issue - how large is the
largest dataset when you try to create it at a monthly level
Weekly Processing
Uses previous 7 daily PDBs Builds the datasets in sequence May be to tape or to disk Volume can quickly become a problem
WTD Processing
Same logic as weekly but run each day At end of week, the same datasets are
created but the resource consumption is spread out
A key is to reduce the number of variables kept
Monthly Processing
Use last 5 weekly and last 7 daily PDBs to construct the previous calendar month.
Logic MUST run on 1st day of month Can be extraordinarily resource intensive Uses tape format datasets on disk to
avoid multiple mounts
MTD Processing
Like the WTD processing, runs each day with a job at the end of the month to create the next MTD PDB.
Reduce the variables even further than the WTD process
Reduce the datasets retained
Processing Cycle
DailyBUILD
MTD Build
WTD Build
DailyPDB
TREND
MTDPDB
!st ofWeek?
!st ofMonth?
Copyto Month
Copyto Week
STOP
Last Week
LastMonth
TRENDBuild
WTDPDB
DailySMF
SPIN
WTD/MTD Processing
Both rely on a new macro VMXG2DTE– You specify input and output Ddnames,
dataset name, by list if appropriate, whether or not to use PROC APPEND, and what cycle and when to start the cycle.
WTD/MTD Processing
%VMXG2DTE( DDIN=MTDPDB, DDOUT=MTDPDB, APPEND=YES, PDB=PDB, DATASET=JOBS, INITIT=M1, DROPPER=var1 var2);
WTD/MTD Processing
%VMXG2DTE( DDIN=WTDPDB, DDOUT=WTDPDB, PDB=PDB, DATASET=RMFINTRV, BYLIST=SYSPLEX SYSTEM STARTIME, INITIT=W2, KEEPER=VAR1 var2 …);
Retention - One Man’s View
Daily PDB is a GDG with 255 generations
Weekly PDB is a GDG with 255 generations with a reduced set of variables and datasets
Monthly PDB is a GDG with 255 generations but a drastically reduced set of variables and datasets
TRENDing
Most of the important data has a TREND component (member TRNDxxxx)
Utilizes VMXGSUM to do summarization
Radical reduction in space for highly summarized data (6 years history in just under 1000 cylinders)
TRENDing
Designed to run weekly but can be done daily by changing the WEEK.xxxxxxxx to PDB.xxxxxxxx in the member
%VMXGSUM(INVOKEBY=TRNDRMFI, INDATA= WEEK.RMFINTRV (IN=INWEEK) TREND.TRNDRMFI,…..);
What Cycles Should You Run?
‘It depends…’– Volume is the key. If volume is small, the
canned structures work fine. As volume grows, these structures become untenable. The run time becomes longer than can be tolerated. A daily job that runs for a day is not practical.
So?? What do you do?
It is after all, just an application not unlike any other application
Apply all of the tricks and techniques you use on applications
Parallelism may be the answer - if it won’t run serially, run it in parallel
So?? What do you do?
Parallel Jobs - example
CMFData
DB2Data
SMF/RMF
TYPE110
TYPEDB2
BUILDPDB
BLDIOPDB
CICSTRAN
DB2PDB
DailyPDB
Daily IO PDB
ASUMUOW ASUMUOW
WTD/MTD
MTDPDB
WTDPDB
So??? What do you do?
No WEEKLY process No MONTHLY process TREND updated daily (which means the
current week is incomplete) Reduced variable counts No detail at the WEEKLY/MONTHLY
level
Parallel Streams
Stream 1 - BUILDPDB– Processes normal BUILDPDB but excludes
all CICS, DB2, and IO related data is the average response time to DASD across
10,000 volumes a significant metric in RMFINTRV? Probably not.
Parallel Streams
Stream 2 - BLDIOPDB– Process IO related data
type 42 type 73 type 74 type 75 type 78 etc.
Parallel Streams
Stream 3 - BLDCISTA– Process CICS Statistics
Stream 4 - DALYDSET– MXG member ANALDSET
Stream 5 - BILDDCOL– Process DCOLLECT data
Parallel Streams
Stream 6 - BUILDTMS– Process TMS Catalog
Stream 7 - BILDDSNS– Combine TMS and DCOLLECT data
Stream 8 - DB2PDB (3 times daily)– Process DB2 data
Parallel Streams
Stream 9 - BLDCISTR (3 times daily)– Process CICS transaction data
Stream 10 - ASUMUOW– Combine CICS/DB2 into Unit of Work
summary
CICS Volume Problem
Processing of CICS/DB2 SMF data into ASUMUOW is time consuming
3 times/day 5-7 hours each– CICS volume is 25GB of SMF data 30.1M
observations (and this was a light day)– DB2 volume is 4.7GB SMF data 2.0M obs
in DB2ACCT Volume is growing 30-40%/year
CICS Volume Problem
CICSTRAN and DB2ACCT must be sorted prior to merge
TAPE IO is bottleneck - CICSTRAN dataset is 5-6 volumes of tape at 10-15 minutes/volume to move data
Data is read/written by data step, by sort then read by data step - 5 full passes
CICSSMF
DB2SMF
CICSTRANSORTED
CICSTRAN
DB2ACCTSORTED
DB2ACCT
TYPE110
ASUMUOW
SORT
SORTTYPEDB2
ASUMUOW
6 Hours
Original Architecture
5-6 tapes 5-6 tapes
5-6 tapes
1-2 tapes
Last Year
CICSSMF
DBACCTSORTED
DB2 SMF
CICSTRANSORTED
ASUMUOW
TYPE110/SORT
TYPEDB2/SORT
ASUMUOW
4.5 Hours
Using VIEW from DATA Step to SORT
Today
CICSSMF
DB2 SMF
ASUMUOW
TYPE110/SORT
TYPEDB2/SORT
ASUMUOW
2.3 Hours
Using VIEW from DATA Step to SORT PIPE from SORT to ASUMUOW
CICSTRANSORTED
DB2ACCTSORTED
fitting
fitting
DB2 Processing
pipe
pipe
Next Step
Solving the CICS Volume Problem
Requires batch pipes for SAS - that will mean at least V8.2 and maybe V9
DB2 Processing must be in 2 steps – the fitting for DB2ACCT can’t be reopened
for input within the same job step. There may be a way to get around this but I haven’t found it yet.
Summary
Whether it is SMF or the processing of SMF, it is just an application program
What works for the standard every day application also works here
Parallel processing solves most of the problems (but not without raising a few issues itself.)
????????????????????????
