insurance client s2 de082 etl technical specification v14.doc

8/12/2019 Insurance Client S2 DE082 ETL Technical Specification v14.doc

1/62

Solvency II 213560877.doc

Insurance Client Logo

INSURANCE CLIENT Solvency II

ETL Technical Specification

For Increment 2

Deliverable Reference DE082

Version 1.14

Status Signed-Off

ile !a"e 213508!!.doc

#ast $dit Date 18"04"2012

#age 1 of 2


2/62


Docu"ent Revie%

!a"e &itle Date

$ites% Lad C&I E'L Lead

(and%ir )a*ala+,rone Solution rc%itect

&ar Carter //lication Delier* &anager

Sue order I& Solution rc%itect

Docu"ent '((roval

!a"e &itle Si)nature Date

&ar Carter //lication Delier* &anager

Version *istory

'ut+or,s- Version R !u"ber '"end"ents/Reason Date

drian ag* 1.0 Initial #u,lication 0"11"2011

drian ag* 1.1 /dated 6it% t%e S# E2E(econciliation

10"11"2011

drian ag* 1.2 /dated ,ased on reie6co++ents

15"11"2011

drian ag* 1.3 /dated 6it% t%e reie6co++ents fro+ &ar Carter

1"11"2011

drian ag* 1.4 dded t%e reconciliations forSO(CE 1-S# and SO(CE1-'DS7 /dated t%e Section ongeneral E'L

18"11"2011

drian ag* 1.5 /dated 6it% t%e reie6

co++ents fro+ #aul '%o+sonStuart $a*nes Sara% 9agstaffand t%e I& offs%ore tea+.

/dated 6it% t%e c%anges fro+t%e C(#(231 - SCS #rocessand $ierarc%* C%anges.

'%e :o, na+es 6ere u/dated.

30"11"2011

drian ag* 1. '%e sections 3.4 and 3.5 6ereu/dated to aggregate to t%e//er(ID leel instead of t%e(ID leel.

'%e sections 3.! and 4.3 6ere

u/dated ,ased on t%e feed,acfro+ t%e off-s%ore tea+.

01"12"2011

drian ag* 1.! /dated t%e E2E (econciliationas /er feed,ac fro+ t%e off-s%ore tea+

05"12"2011

drian ag* 1.8 /dated 6it% t%e reie6co++ents fro+ $ites% Lad7

/dated 6it% feed,ac fro+ t%eoff-s%ore tea+

0!"12"2011

#age 2 of 2


3/62


drian ag* 1. Section !.2 6as u/dated to listt%e :o,s 6%ic% are /o/ulating t%e+etrics ta,le

12"12"2011

drian ag* 1.10 ;et


4/62


&'#$ !&$!&S

1 INTRODUCTION !

1.1 Purpose............................................................................................................................................. 7

1.2 Scope for Increment 2....................................................................................................................... 71.3 SIL Tables......................................................................................................................................... 81.4 External Dependences..................................................................................................................... 8

2 INCRE"ENT 2 ETL #ROCESSIN$ CO"#ONENT O%ER%IE& '

2.1 SIL !"er"e# Da$ram............................................................................................................... ....... %2.2 TDS to TDS !"er"e# Da$ram.............................................................................................. .......... %2.3 &ob Summar' Table........................................................................................................................ 1(

( INTERNAL RECONCILIATION )UILD11

3.1 &obs co"ered n t)s secton............................................................................................................ 113.2 Structure and *appn$s............................................................................................................... ... 11

3.2.1 Data Input.....................................................................................................................................113.2.2 Mapping.................................................................................................................................. ..... 123.2.3 Data Output.................................................................................................................................. 12

3.3 SE(22 + S!,-E 1/TDS -econclaton........................................................................................123.3.1 Description................................................................................................................................... 123.3.2 Job Flow....................................................................................................................................... 133.3.3 Build Specification........................................................................................................................ 13

3.4 SIL((2S!,-E 10TDS0$$re$ate0and0compare....................................................................... 143.4.1 Description................................................................................................................................... 143.4.2 Job Flow....................................................................................................................................... 143.4.3 Build Specification........................................................................................................................ 14

3. SIL((1-eadS!,-E 1Incomn$ ...................................................................................................13..1 Description................................................................................................................................... 1!3..2 Job Flow....................................................................................................................................... 1!3..3 Build Specification........................................................................................................................ 1!

3. SE(23 + S!,-E 1/SP -econclaton........................................................................................173.!.1 Description................................................................................................................................... 1"3.!.2 Job Flow....................................................................................................................................... 1"3.!.3 Build Specification........................................................................................................................ 1"

3.7 SIL((3S!,-E 10SP0$$re$ate0and0compare....................................................................... 183.".1 Description................................................................................................................................... 1#3.".2 Job Flow....................................................................................................................................... 1#3.".3 Build Specification........................................................................................................................ 1$

* SOURCE 1 + SA# E2E RECONCILIATION )UILD2,

4.1 &obs co"ered n t)s secton............................................................................................................ 2(4.2 Structure and *appn$s............................................................................................................... ... 2(

4.2.1 Data Input.....................................................................................................................................2%

4.2.2 Mapping.................................................................................................................................. ..... 214.2.3 Data Output.................................................................................................................................. 21

4.3 SE(21 + S!,-E 1/SP E2E -econclaton................................................................................214.3.1 Description................................................................................................................................... 214.3.2 Job Flow....................................................................................................................................... 224.3.3 Build Specification........................................................................................................................ 22

4.4 SIL((40Load0SP0Postn$s..........................................................................................................234.4.1 Description................................................................................................................................... 234.4.2 Job Flow....................................................................................................................................... 244.4.3 Build Specification........................................................................................................................ 24

#age 4 of 2


5/62


4. SIL((0Load0SP0Errors.............................................................................................................. 244..1 Description................................................................................................................................... 244..2 Job Flow....................................................................................................................................... 244..3 Build Specification........................................................................................................................ 2

4. SIL((0Load0SP0&ournal0and0Detal0Errors............................................................................. 24.!.1 Description................................................................................................................................... 24.!.2 Job Flow....................................................................................................................................... 24.!.3 Build Specification........................................................................................................................ 2!

4.7 SIL((70E2E0$$re$ate0and0compare..........................................................................................24.".1 Description................................................................................................................................... 2!4.".2 Job Flow....................................................................................................................................... 2"4.".3 Build Specification........................................................................................................................ 2"

- TDS TO TDS SNA#S.OT CO"#ONENT )UILD (,

.1 &obs co"ered n t)s secton............................................................................................................ 3(.2 Structure and *appn$s............................................................................................................... ... 3(

.2.1 Data Input.....................................................................................................................................3%.2.2 Mapping.................................................................................................................................. ..... 31.2.3 Data Output.................................................................................................................................. 31

.3 SE(18 + TDS o"er Snaps)ot for S!,-E 1............................................................................... 31

.3.1 Description................................................................................................................................... 31.3.2 Build Specification........................................................................................................................ 31

.4 SE(1% + TDS o"er Snaps)ot for S!,-E 2............................................................................... 32.4.1 Description................................................................................................................................... 32.4.2 Build Specification........................................................................................................................ 32

. SE(2( + TDS Dal' *easures.........................................................................................................33..1 Description................................................................................................................................... 33..2 Build Specification........................................................................................................................ 33

. TDS((10et0SS0for0S!,-E 1................................................................................................33.!.1 Description................................................................................................................................... 33.!.2 Job Flow....................................................................................................................................... 33.!.3 Build Specification........................................................................................................................ 34

.7 TDS((20et0SS0for0S!,-E 2................................................................................................3

.".1 Description................................................................................................................................... 3!.".2 Job Flow....................................................................................................................................... 3!.".3 Build Specification........................................................................................................................ 3!

.8 T-((1 + S)ared ontaner for lass Structure loo5ups...............................................................4(.#.1 Description................................................................................................................................... 4%.#.2 &ontainer Flow............................................................................................................................. 41.#.3 Build Specification........................................................................................................................ 41

.% TDS((30Load0o"er0Snaps)ot ....................................................................................................42.$.1 Description................................................................................................................................... 42.$.2 Job Flow....................................................................................................................................... 42.$.3 Build Specification........................................................................................................................ 42

.1( TDS((40alculate0Dal'0*easures............................................................................................43.1%.1 Description................................................................................................................................ 43

.1%.2 Job Flow....................................................................................................................................43.1%.3 Build Specification..................................................................................................................... 43.11 TDS((0Load0Dal'0*easures...................................................................................................44

.11.1 Description................................................................................................................................ 44.11.2 Job Flow....................................................................................................................................44.11.3 Build Specification..................................................................................................................... 44

/ #OLIC0 .EADER U#DATE TO INCRE"ENT 1 SIL TO TDS O)*/

.1 &obs co"ered n t)s secton............................................................................................................ 4.2 Structure and *appn$s............................................................................................................... ... 4

#age 5 of 2


6/62


!.2.1 Data Input.....................................................................................................................................4!!.2.2 Mapping.................................................................................................................................. ..... 4!!.2.3 Data Output.................................................................................................................................. 4"

.3 SE(17 + Load Transacton Tables.................................................................................................. 47!.3.1 Description................................................................................................................................ .. 4"!.3.2 Job Flow...................................................................................................................................... 4"!.3.3 Build Specification.......................................................................................................................4"

! $ENERAL ETL #ROCESSIN$ *'

7.1 Sc)eduln$...................................................................................................................................... 4%".1.1 '2' (econciliation Batc) Flow............................................................................................. ....... 4$".1.2 SI* Batc) Flow............................................................................................................................. %".1.3 +DS Batc) Flow.......................................................................................................................... . 1".1.4 Batc) I,ple,entation Details....................................................................................................... 1

7.2 &ob -ecordn$................................................................................................................................. 27.3 Error *ana$ement.......................................................................................................................... 77.4 *antann$ reference data.............................................................................................................. %7. Parameter *ana$ement..................................................................................................................%7. 6le S'stem Structure...................................................................................................................... 17.7 6TP 6le locatons for De" and Test................................................................................................ 2

#age of 2


7/62


1 Intro34ction

11 #4rpo5e

'%e /ur/ose of t%is docu+ent is to /roide t%e detailed leel tec%nical s/ecification for t%e E'Lco+/onents in sco/e of Incre+ent 2 of t%e IS(CE CLIE' Solenc* II (elease 0.1 solution. '%isincludes /arts of t%e Single Integration La*er ASILB and t%e load to t%e Sna/s%ot and &easures 'a,les int%e 'ransactional Data Store A'DSB. '%e docu+ent 6ill descri,e t%e flo6 of t%e E'L /rocesses 6%ic% 6ill,e used ,* t%e ,uild tea+ 6%en deelo/ing E'L :o,s.

12 Scope for Increment 2

'%e diagra+ ,elo6 is an oerie6 of all t%e E'L co+/onents 6it%in sco/e of (elease 0.1 and t%ose 6it%int%e red ,o?es are in sco/e of incre+ent 2.

'%e e* functional areas are

Internal (econciliation and Integrit* C%ecing

SO(CE 1 > S# E2E (econciliation

Classification follo6ed ,* 'DS Load for SO(CE 1 and SO(CE 2 to Coer Sna/s%ot ta,le

Calculations follo6ed ,* 'DS Load for SO(CE 1 and SO(CE 2 to Dail* &easures

#age ! of 2


8/62


1( SIL Ta6le5

'%ere are a nu+,er of S@L Serer ta,les t%at are used ,* t%e E'L :o,s during t%e /rocessing. '%issc%e+a is ,eing +aintained se/aratel* fro+ t%e +ain 'DS sc%e+a and 6ill also ,e t%e su,:ect of ersioncontrol.

1* E7ternal Depen3encie5

'%e follo6ing e?ternal de/endencies %ae ,een identified

DataStage Serer aila,ilit*

Data,ase aila,ilit*

Source ile aila,ilit*

#age 8 of 2


9/62


2 Increment 2 ETL #roce55in8 Component

Overvie9

21 SIL Overvie9 Dia8ram

'%e diagra+ ,elo6 s%o6s t%e tec%nical sco/e for t%e SIL for Incre+ent 2 ,roen do6n into nu+,ered :o,s.ll E'L :o,s 6it%in t%e solution 6ill ,e /refi?ed 6it% SE ASolenc* II E'LB. '%ere is one :o, seuence for t%eSO(CE 1 - S# E2E reconciliation and t6o :o, seuences for t%e SO(CE 1-S# and SO(CE 1-'DSreconciliations.

S-#

(econciliation

1iles

I!& 6

S-# E2E

(econciliation

S$021

22 TDS to TDS Overvie9 Dia8ram

'%e diagra+ ,elo6 s%o6s t%e tec%nical sco/e for t%e 'DS to 'DS load for Incre+ent 2 ,roen do6n into

nu+,ered :o,s.

#age of 2


10/62


Incre"ent 2

I!&

SI11

1iles

SIFF to TDS

Load 'ransaction

'a,les

S$017

(e:ected SI11

(ecords

SI11 1ile (ecei/t

S$015

SI11 1ile

)alidation

S$016

(e:ected SI11

1iles

'DS Co0er

Sna/s%ot for -IS

S$018

'DS Dail*

&easures A-ISB

S$020

'DS Co0er

Sna/s%ot for

5E(IS

S$013

'DS Dail*

&easures

A5E(ISB

S$020

2( o6 S4mmary Ta6le

Fo, u+,er Aa+eB Descri/tion

SE018 Fo, Seuence used to get t%e SCS and load t%e'DS S#reconciliation

SIL001(eadSO(CE 1Inco+ing Fo, 6%ic% 6rites t%e SO(CE 1 Inco+ing data into adataset.

#age 10 of 2


11/62


( Internal Reconciliation )4il3

(1 o65 covere3 in thi5 5ection

'%e diagra+ ,elo6 s%o6s t%e :o, seuences and :o,s used to reconcile data ,et6een SO(CE 1 andS# and ,et6een SO(CE 1 and 'DS.

(2 Str4ct4re an3 "appin85

(21 Data Inp4t

In(ut !a"e Descri(tion

SIL


12/62



SIL


13/62


((2 o6 Flo9

((( )4il3 Specification

DataSta)eco"(onent DataSta)e co"(onent ty(e/na"e Descri(tion

Seuencena+e

SE022


14/62


(* SIL,,2SOURCE 1:TDS:A88re8ate:an3:compare

(*1 De5cription

'%is :o, aggregates t%e data fro+ SO(CE 1 and t%e data sent to 'DS and co+/ares t%e results.

(*2 o6 Flo9

(*( )4il3 Specification

DataSta)eco"(onent

DataSta)e co"(onent ty(e/na"e Descri(tion

Fo, na+e SIL002SO(CE1


15/62


DataSta)eco"(onent


nu+,er for t%e current run A:o, /ara+eterB

In/ut Stage ODC Connector

ASIL


16/62


(- SIL,,1Rea3SOURCE 1Incomin8

(-1 De5cription

'%is :o, :oins and aggregates t%e data fro+ SO(CE 1 and 6rites it to a dataset.

(-2 o6 Flo9

(-( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL001(eadSO(CE 1Inco+ing '%is :o, :oins and aggregates t%e data fro+

SO(CE 1 and 6rites it into a dataset6%ic% can ,e used in t%e reconciliation/rocess.

In/ut Stage ODC Connector ASIL


17/62


DataSta)eco"(onent


#ara+eters Serial nu+,er

D a+e

D ser

D #ass6ord

'%e seuence 6%ic% is /art of t%e filena+esfor 6%ic% t%e reconciliation /rocess is done

Data,ase na+eData,ase user na+e

Data,ase /ass6ord

(/ SE,2( + SOURCE 1SA# Reconciliation

(/1 De5cription

'%is :o, seuence re/resents t%e oerall flo6 needed to /erfor+ t%e SO(CE 1-S# (econciliation. '%e

co+/onents and actiities in t%e seuence are detailed ,elo6 in t%e ,uild s/ecification ta,le.

(/2 o6 Flo9

(/( )4il3 Specification

DataSta)eco"(onent


Seuencena+e

SE023


18/62


DataSta)eco"(onent


Co++and e+/t*. If *es t%e reconciliation issuccessful.

otificationctiities

otifications otif* t%e /ersons res/onsi,le on t%esuccess or t%e failure of t%e reconciliation/rocess.

Include t%e reconciliation out/ut file as anattac%+ent.

#ara+eters Seuence nu+,er

D a+e

D ser

D #ass6ord

E+ail addresses

'%e seuence 6%ic% is /art of t%e filena+esfor 6%ic% t%e reconciliation /rocess is done

Data,ase na+e

Data,ase user na+e

Data,ase /ass6ord

'%e e+ail addresses of t%e /eo/le to ,enotified in case of errors

(! SIL,,(SOURCE 1:SA#:A88re8ate:an3:compare

(!1 De5cription

'%is :o, aggregates t%e data fro+ SO(CE 1 and t%e data sent to S# and co+/ares t%e results.

(!2 o6 Flo9

K

#age 18 of 2


19/62


(!( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL003SO(CE1


20/62


* SOURCE 1 + SA# E2E Reconciliation )4il3

*1 o65 covere3 in thi5 5ection

'%e diagra+ ,elo6 s%o6s t%e Fo, Seuences and t%e Fo,s used to reconcile t%e data /osted ,* S# 6it%t%e data fro+ SO(CE 1 sent to 'DS and S#.

S$021 S' $2$ Reconciliation

Load S-##ostings

SI#00

Load S-# Errors

SI#005S-#

(econciliation

1iles

I!& 6

Load S-#

)alidation ErrorsAFournal and

DetailB

SI#006

-ggregate and

Co+/are

SI#007

*2 Str4ct4re an3 "appin85

*21 Data Inp4t


aiennnnnnsa/.dat ile containing S# /ostings fro+ t%e S#reconciliation data. '%e file for+at is docu+ented int%e functional design docu+ent referenced as (D1.

aiennnnnnerr.dat ile containing S# errors fro+ t%e S#reconciliation data. '%e file for+at is docu+ented int%e functional design docu+ent referenced as (D1.

aiennnnnne01 ile containing S# re:ected :ournal and detailrecords fro+ t%e S# reconciliation data. '%e filefor+at is docu+ented in t%e functional designdocu+ent referenced as (D1.

#age 20 of 2


21/62


*22 "appin8

4a((in) Descri(tion

Source files to SIL


22/62


*(2 o6 Flo9

*(( )4il3 Specification

DataSta)eco"(onent


Seuencena+e

SE021


23/62


DataSta)eco"(onent


E?ecuteCo++and

C&D Scri/t 6%ic% deletes t%e records 6it% t%esa+e filena+e fro+ t%e S# ta,les inSIL.

Fo, ctiit* SIL004


24/62


**2 o6 Flo9

**( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL004


25/62


*-( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL005


26/62


*/( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL00


27/62


*!2 o6 Flo9

*!( )4il3 Specification

DataSta)eco"(onent


Fo, na+e SIL00!


28/62


DataSta)eco"(onent


filters t%e records on t%e serial nu+,er eualsoutgoing


29/62


DataSta)eco"(onent


out/ut lin.

Out/ut Stages Seuential ile seuential file 6%ic% 6ill contain 0"1 on t%e

first line to indicate t%e reconciliation 6assuccessful or not. It 6ill also contain t%ereconciliation figures A,ot% count and su+ ofa+ountB. 9ill ,e read fro+ inside t%e :o,seuence.

#ara+eters ile Location

Source filena+e serial nu+,er

D a+e

D ser

D #ass6ord

'%e /at% 6%ere t%e out/ut file is to ,e 6ritten

#art of t%e filena+e used to identif* t%e files

Data,ase na+e

Data,ase user na+e

Data,ase /ass6ord

#age 2 of 2


30/62


- TDS to TDS Snap5hot Component )4il3

-1 o65 covere3 in thi5 5ection

'%e diagra+ ,elo6 s%o6s t%e Fo, Seuences and t%e Fo,s used to /o/ulate t%e 'DS Coer Sna/s%ot andt%e 'DS Dail* &easures ta,les. or 'DS Coer Sna/s%ot t%ere are se/arate :o, seuences for eac%source s*ste+s SO(CE 1 and SO(CE 2 ,ut t%e* are calling t%e sa+e :o,s e?ce/t for t%edeter+ination of t%e SCS 6%ere t%e logic and t%e fields used are co+/letel* different. or t%e 'DS Dail*&easures t%ere is onl* one :o, seuence. '%e :o,s and :o, seuences 6%ic% are Hs%ared ,* SO(CE 1and SO(CE 2 6ill %ae a /ara+eter to s/ecif* t%e Source S*ste+ and t%e alue of t%is /ara+eter s%ould,e /art of t%e dataset filena+e 6%ere a//ro/riate.

-2 Str4ct4re an3 "appin85

-21 Data Inp4t


'DS #olic* 'ransaction 'a,le '%e /ri+ar* fact ta,le %olding transaction leel

#age 30 of 2


31/62



/re+iu+ data.

-22 "appin8

4a((in) Descri(tion

'DS #olic* 'ransaction to 'DS Coer Sna/s%otand 'DS Dail* &easures

&a//ing t%at details %o6 t%e fields in t%e sourceta,les are +a//ed into t%e 'DS Coer Sna/s%ot andDail* &easures 'a,les.

'%is can ,e found in t%e Source to 'arget &a//ings/reads%eet t%at acco+/anies t%is docu+ent A(D8B.

-2( Data O4tp4t

ut(ut !a"e Descri(tion

'DS Coer Sna/s%ot '%is ta,le %olds coer data at t%e lo6est aaila,leleel toget%er 6it% t%e SCS code a//ro/riate to t%e/olic*. #olic* ID allo6s lining ,ac u/ to t%e #arent/olic* of 6%ic% t%is coer is a /art.

'DS Dail* &easures '%is ta,le %olds nearned #re+iu+ (esere A#(Band Earned #re+iu+ AE#B data for eac% indiidualcoer. It is recalculated on a sc%eduled ,asis Aeit%erdail* or +ont%l*B and t%e date of calculation is

recorded. &easures are alid onl* on t%at date. '%edail* c%ange in E# and #( is %eld on t%is ta,le tofacilitate ad-%oc calculation.

-( SE,1; + TDS Cover Snap5hot for SOURCE 1

-(1 De5cription

'%is :o, seuence e?ecutes t%e :o,s 6%ic% /o/ulate t%e 'DS Coer Sna/s%ot for t%e SO(CE 1 sources*ste+.

-(2 )4il3 Specification

DataSta)eco"(onent


Fo, Seuencena+e

SE018


32/62


DataSta)eco"(onent


SO(CE 1 source s*ste+.

Fo, ctiit* 'DS001


33/62


-- SE,2, + TDS Daily "ea54re5

--1 De5cription

'%is :o, seuence e?ecutes t%e :o,s 6%ic% /o/ulate t%e 'DS Dail* &easures for t%e SO(CE 1 and

SO(CE 2 source s*ste+s. '%e :o, seuence %as a /ara+eter to s/ecif* t%e Source S*ste+ and its alueit is /assed to t%e e?ecuted :o,s. '%e transfor+ation logic does not de/end on t%e source %ence t%ere is noneed to ,uild se/arate :o,s or :o, seuences for eac% source s*ste+ as in t%e 'DS Coer Sna/s%ot case.

--2 )4il3 Specification

DataSta)eco"(onent


Seuencena+e

SE020


34/62


-/( )4il3 Specification

DataSta)eco"(onent


Fo, na+e 'DS001


35/62


DataSta)eco"(onent


to-target +a//ings. dd t%e rangecondition for t%e date Asna/s%otdate ,et6een start date and end

dateB to +ae sure t%at t%e loou/returns t%e latest record. C%anget%e /artitioning +et%od on t%ereference lin to HEntire.

Container S%ared ContainerAC'(


36/62


-! TDS,,2:$et:SCS:for:SOURCE 2

-!1 De5cription

'%is :o, 6ill +a/ t%e SO(CE 2 #olic* 'ransaction data to SCS.

-!2 o6 Flo9

-!( )4il3 Specification

DataSta)eco"(onent


Fo, na+e 'DS002


37/62


DataSta)eco"(onent


euals HSO(CE 2. '%is alue iss/ecified in a /ara+eter. lso filteronl* ne6 transactions since t%e last

load ,* reading t%e ro6s 6it% t%e/olic*


38/62


DataSta)eco"(onent


dateB to +ae sure t%at t%e loou/returns t%e latest record. C%anget%e /artitioning +et%od on t%e

reference lin to HEntire.

#rocessing Stage Loou/AL#


39/62


DataSta)eco"(onent


#rocessing Stage Loou/ AL#


40/62


DataSta)eco"(onent


Out/ut Stage Dataset ileADS


41/62


-;2 Container Flo9

-;( )4il3 Specification

DataSta)eco"(onent


In/ut Stage ODC Connector StageAD


42/62


-' TDS,,(:Loa3:Cover:Snap5hot

-'1 De5cription

'%is :o, loads t%e 'DS


43/62


-1, TDS,,*:Calc4late:Daily:"ea54re5

-1,1 De5cription

'%is :o, calculates all t%e Earned #re+iu+ and nearned #re+iu+ (esere dail* +easures.

-1,2 o6 Flo9

-1,( )4il3 Specification

DataSta)e

co"(onent


Fo, a+e 'DS004


44/62


DataSta)eco"(onent


#ara+eters Dataset Location

D a+e

D SE(

D #9D

Source S*ste+

Calculation Date

'%e /at% 6%ere t%e datasets are stored.

Data,ase na+e

Data,ase ser a+e

Data,ase #ass6ord

'%e source s*ste+ na+e.

'%e date t%e calculations are +ade for.

-11 TDS,,-:Loa3:Daily:"ea54re5

-111 De5cription

'%is :o, loads t%e 'DS


45/62


DataSta)eco"(onent


/ara+eteriMed stored /rocedure 6%ic%s%ould ,e called as Hefore S@L on t%eODC Stage. '%ere are t6o actions

6%ic% need to ,e done

- c%ec if t%e data is alread*loaded for t%e calculation dateand if so ,ased on t%e alue of a/ara+eter eit%er dro/ t%e/artition and create it again orsi+/l* add data to t%e sa+ecalculation date

- dro/ t%e /artitions 6%ic% areolder t%an 3 calendar +ont%s/lus one 6ee e?ce/t t%e/artitions for t%e last da* in eac%

+ont% as 6e need to ee/ t%ree+ont%s rolling of %istor* /lusend-+ont% figures

#ara+eters Dataset Location

D a+e

D SE(

D #9D

Source S*ste+

'%e /at% 6%ere t%e datasets are stored.

Data,ase na+e

Data,ase ser a+e

Data,ase #ass6ord

'%e source s*ste+ na+e.

#age 45 of 2


46/62


/ #olicy .ea3er Up3ate to Increment 1 SIL to

TDS =o6

/1 o65 covere3 in thi5 5ection

'%is section s%o6s t%e u/date needed to ,e done to t%e incre+ent 1 SE01! :o, to load t%e 'DS #olic*$eader ta,le.

Load 'ransaction

'a,les

S$017

(e:ected SI11

(ecords

/2 Str4ct4re an3 "appin85

/21 Data Inp4t


e6 in/ut 'DS


47/62


/2( Data O4tp4t

ut(ut !a"e Descri(tion

e6 out/ut 'DS


48/62


DataSta)eco"(onent


t%e #olic* ID.

If t%e loou/ fails go to t%e re:ect lin and

continue 6it% t%e Surrogate e*;enerator stage.

If t%e loou/ is successful go to t%eunnel stage.

#rocessingStage

Surrogate e* ;enerator A#olic* IDB 9%en t%e loou/ to t%e #olic* $eader isnot successful t%is stage generates ane6 alue to ,e used as a #olic* ID.

#rocessingStage

Co/* Co/ies t%e in/ut to t6o out/uts. Dro/ t%eunnecessar* colu+ns on eac% out/ut.

#rocessing

Stage

unnel nions t%e records

Out/ut Stage ODC Connector 9rites to t%e 'DS #olic* $eader ta,le.

#age 48 of 2


49/62


! $eneral ETL #roce55in8

'%is section descri,es t%e ;eneral E'L /rocessing ele+ents t%at su//ort t%e functional as/ects of t%es*ste+ descri,ed in t%e sections a,oe.

!1 Sche34lin8

'%is section s%o6s t%e ,atc% sc%edule and flo6 for ,ot% SIL N 'DS. It t%en also /roides details a,out %o6t%is 6ill ,e i+/le+ented 6it%in DataStage.

!11 E2E Reconciliation )atch Flo9

'%e diagra+ a,oe s%o6s t%e E2E (econciliation atc% lo6. '%e /rocess 6ill ,e triggered ,* t%e/resence of t%e last file e?/ected fro+ S# as defined in t%e corres/onding section a,oe. '%is /rocess isinde/endent of t%e ot%er flo6s.

#age 4 of 2


50/62


!12 SIL )atch Flo9

'%e diagra+ a,oe %ig%lig%ts t%e Incre+ent 2 flo6s 6%ic% 6ill ,e /laced inside t%e oerall SIL ,atc%.

#age 50 of 2


51/62


!1( TDS )atch Flo9

'%e diagra+ a,oe %ig%lig%ts t%e Incre+ent 2 flo6s 6%ic% 6ill ,e /laced in t%e oerall 'DS atc%.

!1* )atch Implementation Detail5

'%e ,atc% 6ill ,e i+/le+ented as follo6s

'i+e ased Seuences

'%ese 6ill ,e sc%eduled to run in t%e DataStage Sc%eduler

ile ased Seuences

'%ese 6ill ,e sc%eduled to run in t%e DataStage Sc%eduler %o6eer t%e* 6ill ,e controlled ,* file6atc%ers oer a /eriod of ti+e. E.g. t%e SO(CE 1 seuence 6ill ,e started at 3a+ ,* t%esc%eduler and t%en it 6ill continue to 6atc% for files until !a+.

&anual 'riggering

S%ould t%ere ,e a reuire+ent to trigger a seuence +anuall* t%is can ,e done fro+ DataStageDirector.

#arallel N &ulti/le ile #rocessing

'%e SO(CE 2 N SO(CE 1 seuences can run in /arallel %o6eer t%e* 6ill onl* /rocess onefile at a ti+e. '%erefore if +ulti/le files are /resent for SO(CE 1 or SO(CE 2 t%e* 6ill ,e/rocessed seuentiall* to +aintain t%e integrit* of t%e (ID allocation /rocess.

(estarta,ilit*

'%e seuences 6ill ,e designed to ,e restarta,le suc% t%at if t%ere is a s*ste+ failure t%e* can ,erestarted and 6ill /ic u/ fro+ 6%ere t%e* sto//ed. '%e* 6ill also %ae controls in /lace t%at c%ec6%et%er t%e seuence can run > e.g. are t%e inter+ediate ta,les clear is a seuence alread*running is t%e seuence in a runna,le state Ai.e. not a,ortedB.

#age 51 of 2


52/62


!2 o6 Recor3in8

Fo, recording is t%e co+/onent t%at logs infor+ation a,out t%e e?ecuting DataStage :o, suc% as start ti+eduration and olu+e of data /rocessed and 6%et%er t%e :o, co+/leted successfull* or failed.

Eac% :o, in DataStage is identified ,* a uniue ID and eac% e?ecution of a :o, is identified ,* a uniue

,atc% nu+,er. atc% nu+,ers +a* ,e t%e sa+e for +ore t%an one :o, for e?a+/le a series of :o,s t%at+ae u/ a /rocessing sc%edule.

'%e ta,le ,elo6 details t%e E'L Control 'a,les

&R#D$!

olu"n na"e Data &y(e 9ey Descri(tion

Fo,


53/62


t%at deli,eratel* a,ort 6%ile

failed is 6%ere t%e :o, 6asforced to ter+inate fore?a+/le s*ste+ failure.

&R#$:$4$&RI

olu"n na"e Data &y(e 9ey Descri(tion

&etric


54/62


'%e follo6ing ta,le s%o6s t%e :o, seuences and t%e :o,s 6%ic% 6ill record t%e infor+ation to t%eC'(L


55/62


ob se;uence na"e ob na"e 4etrics recorde

SE00


56/62


ob se;uence na"e ob na"e 4etrics recorde

SE01!


57/62


!( Error "ana8ement

reuire+ent of t%e s*ste+ is to log all errors t%at occur so t%at t%e* can ,e anal*sed and acted on. '%einfor+ation to ,e logged includes

'%e E?ecution ID of t%e DataStage :o, t%at rSource 1ed t%e error

'%e Error Code

#rocess or &odule reference to identif* 6%ere in t%e :o, t%e error occurred

Error details. '%e error code defines t%e t*/e of error for e?a+/le Inalid ile or+at and t%is

field /roides details for 6%* t%e error 6as generated

9%en recording errors t%e descri/tion +ust /roide full details of t%e error as in t%e e?a+/le a,oe. '%isdata is recorded in t%e Error Log ta,le.

'%e ta,les t%at su//ort Error Logging are detailed ,elo6

&R#$RRR ,Static-

olu"n na"e Data &y(e 9ey Descri/tion

E((O(


58/62


&R#$RRR#

insurance client s2 de082 etl technical specification v14.doc

Documents