2013_01_29_5108_3__DataStage_HOL.pdf

8/15/2019 2013_01_29_5108_3__DataStage_HOL.pdf

1/25

© 2011 IBM Corporation

Information Management

DataStage Hands on Lab

2013-01-30


2/25


3/25



Set6p prior to Lab

Config6ration for DB2& C/assi 4ed& CDC

+ DB2

< Create tab/es and /oad data

< Bind DB2 pa=age >db2 bind db2!e5L$9s?/bnd datetime iso b/o=ing a//@

< Cata/og remote database # DSNT % in /oa/ DB2 instane # db2inst1 % + C/assi 4ederation

< Define data so6res in Data Serer

< Define DBC DS$ on !Lin6. # IMSD %

+ CDC

< Config6re CDC so6re >!S@ and target >DS@ agents< Create s6bsriptions and define tab/e mappings

< Create Boo=mar= DS$

DataStage proAet artifats

+ Import tab/e definitions

+ Create /ab parameter set


4/25



7.erise 0 Simp/e *ara//e/ ob

Create a simp/e para//e/ Aob 6sing o" enerator and *ee= stages

Modify para//e/ism by 'anging onfig fi/e and para//e/ be'aior in Aob design


5/25



Step 1 Create Aob

Create Aob as s'o"n

;dd *arameter Set DSEL;BE*;MS

Set $6mber of eords to 10&000&000

Load t'e F;D7E1 s'ema Sae and Compi/e ob


6/25



Step 2 6n Aob

6n Aob 6sing Designer 6n b6tton

G$otie t'e ,fan ion indiating se?6entia/ to para//e/ be'aior

Modify o" enerator to r6n in para//e/

+ Stage ;daned tab Set Execution mode to Parallel

G$otie t'e /in= ion 'anges to indiate para//e/ to para//e/ be'aior "it' no repartitioning

6n t'e Aob again 20M ro"s "i// be proessed in >ro6g'/y@ t'e same amo6nt of time


7/25



7.erise 1 ead and /oad DB25

Create a Aob to read and "rite data tofrom a DB2 tab/e

Fab/e partitioning

Different met'ods for /oading data

+ Data SetJ

+ Bat' *ipes

+ USS pipes

J Met'od 6sed in /ab


8/25



Step 1 Create Aob >1@

Create Aob as s'o"n


Set $6mber of eords to 1&000&000

Load t'e F;D7E1 s'ema erride generator defa6/ts for FEID

enerator properties for FEID


9/25




Set *arameters in DB2 Connetor as s'o"n

Sae and Compi/e ob


10/25



Step 2 6n Aob


F'is Aob performs a se?6entia/ /oad into t'e tab/eGIn t'e Aob r6n /og& yo6 an see o6tp6t from t'e DB2 /oad 6ti/ityGF'e DB2 /oad 6ti/ity time is in/6ded in t'e Aob ,start time

;n M:S data set is reated named I$00000 "it' t'e data for /oading


11/25



Step 3 Modify Aob for para//e/ /oad

Set *artitioning type to ,DB2 ConnetorGF'is te//s t'e onnetor to ?6ery t'e database and 6se t'e tab/e partitioning to

para//e/i!e t'e /oad

Sae and Compi/e ob


12/25



Step K 6n Aob


Sine t'e tab/e 'as 2 partitions& a para//e/ /oad into t'e tab/e is 6sedGIn t'e Aob r6n /og& yo6 an see o6tp6t from t'e DB2 /oad 6ti/ity

M:S data sets are reated for ea' partition I$00001& I$00002


13/25



Step DB2 to DB2

ep/ae o" enerator "it' DB25 stageGDB25 stage may not be on t'e pa/ette& an add from epository ie"

Set properties as aboeGead Met'od(Fab/e "i// ?6ery t'e ata/og and read t'e tab/e in para//e/ based

in t'e tab/e partition definition

Compi/e and r6n t'e Aob


14/25



7.erise 2 !S 4i/e and C/assi 4ederation

Create a Aob to read from a C/assi 4ed so6re and "rite to an M:S data set


15/25




Create Aob as s'o"n


Config6re C/assi 4ederation Stage as s'o"n

Load LF*SU**LI7 tab/e definition


16/25




C44 stage is 6sed to prepare t'e data for t'e !S4i/e stage

Inp6t o/6mns are mapped to sing/e o6tp6t o/6mn

Use eord options and eords tab to speify formatting

eord si!e is a/6/ated based on t'e stage s'ema


17/25




Config6re !S4i/e stage as s'o"n

Inp6t fie/d represents t'e entire reord

Sae and ompi/e


18/25



Step 2 6n Aob


M:S data set "i// be reated if it doesnt e.ist& oer"ritten if it does


19/25



7.erise 3 CDC to DataStage

Create a Aob to read from a CDC s6bsription and "rite to a DB2 LU8 target tab/e

In order to maintain transationa/ integrity a// stages m6st r6n se?6entia//y


20/25



Step 1 Import s6bsription Aob

CDC Management onso/e "i// generate a base DataStage Aob for t'e s6bsription

Import t'e Aob into Designer

Basi s6bsription information is fi//ed in Lin=s are reated for t'e boo=mar= and ea' tab/e being rep/iated


21/25



Step 2 C6stomi!e Aob >1@


Insert transformer stage on t'e F;D7 /in=

Config6re CDC stage as s'o"n + Boo=mar= tab/e 'as been pre-reated for ea' s6bsription

N6serOBPM;PF;BL7& so no 'ange needed


22/25




Map so6re o/6mns as s'o" to reate o6tp6t o/6mns

7dit e.pression for B747EFEID and B747EFEDFS to s6bstit6te p/ae'o/der

a/6es "'en not proessing an 6pdate

Set ,7.e6tion mode to Se?6entia/ to maintain transationa/ integrity


23/25




Set DB2 Connetion properties as s'o"n

$o 'anges re?6ired for Boo=mar= /in=


24/25



Step 3 Start t'e s6bsription

Use t'e CDC Management onso/e to start t'e s6bsription

GF'e s6bsription "i// stay in ,Starting state 6nti/ t'e DataStage Aob is started

GCan onfig6re t'e Aob to start a6tomatia//y

Start t'e Aob 6sing Designer

Go"s "i// start to f/o" do"n t'e boo=mar= /in=


25/25



Step K ;dd data to t'e so6re tab/e

6n t'e Aob dE'angeEsr to insert reords into t'e so6re tab/e

F'e CDC Aob "i// proess t'e apt6red 'ange ro"s

8'en t'e s6bsription is stopped& t'e DS Aob "i// stop

Documents

2013_01_29_5108_3__DataStage_HOL.pdf