siebel ucm

© Tata Consultancy Services ltd. December 1, 2010 1

UCMCDI Integrating Application

2

Text in blue and black can be changedPositions in Blue text cannot be alteredText in black can be altered in position and sizes if need beText in yellow ochre is meant for legal matters and updates

COURSE AGENDA

Day 1 Module 1 : Review of CDI Architecture Module 2 : Overview of Application Integration Module 3 : Initial Data Loads into UCM Module 4 : The UCM Batch Workflow and SDH tables

Day2 Module 5 : UCM Messaging and CRMML Module 6 : UCM Server Party Package Workflows Module 7 : The UCM Batch Management Workflows Module 8 : Publish and Subscribe Module 9: Wrap-up

3


Module 1: Review of Siebel CDI Architecture

4


The CDI Solution The Customer Data integration(CDI) solution consists of three

products

CDI

SDQUCM UAN

5


UCM = Universal Customer Master What is UCM? A Siebel horizontal application which is the central master

database/application for storing any and all customer information . What it does?

1. Acts as a central repository of customer data. 2. Keeps customer information in synch with other systems in the enterprise, thus giving a “360° real-time true picture” of the customer.

In a typical large enterprise, you have hundreds of various proprietary and legacy applications which deal with different parts of the customer information , some deal with service data, some with sales data, some others purely with demographics and contact data. UCM has the capability to synchronize and bring together all this data to give a single unified view of the customer as a whole.

6


SDH = Source Data and History tables

The central components of the CDI solution are the Source Data and History (SDH) tables associated with UCM

1. Provide a mechanism to maintain audit-trace of changes happening to the customer record.

2. Provide a staging area for data quality. External systems first integrate with these SDH tables when

sending data to UCM, UCM will then in turn use it’s data quality rules to sort out the “good” updates from the “bad” ones and apply the “good” ones to the base tables while storing the “bad” ones too for audit purposes.

E.g. In an enterprise, suppose you have a .NET based sales application and a legacy service application, both storing the customer’s contact number. If the enterprise feels that it’s service department is more likely to have a more current contact number of the customer, you can define the service application with a higher confidence level for updating this field. In cases where both applications are sending simultaneous updates to this field, both updates will be stored in the SDH tables, but the one from the service application will get applied to the base table.

7


UCM Data Model Is an extension of Siebel Data model

Source, Data and history tables are added. UL

BL

DL

Applet View Screen ApplicationList Column/Control

Field

SDH

Column

BC BO

SDH

table

.

8


Source Data and History types SDH data contains several types of records distinguished by a

Type value— Each record is one of the following types:

1. History2. Source3. Merged4. Cleansed5. Queued6. Unmerge7. Batch

A single transaction typically results in the creation of multiple Source data and history records.

9


Source Data and History : Data schema

Provide the infrastructure necessary to support child structures Example : addresses for contacts

S_ORG_EXT

S_UCM_ORG_EXT

S_UCM_ORG_CHLD

S_UCM_CONTACT

S_UCM_CON_CHLD

S_CIF_EXT_SYST

S_CONTACT

Source data History Table

Base table

Source Data History Child table

External System information table

10


UCM Application Integration Revolves around integrating to these SDH tables First basic CDI configuration must be performed

1. Systems registered2. Data quality configured3. Survivorship configured

11


CDI Configuration External System Registration

Data Quality Configuration

Survivorship Rules

12


External System Registration Every operational application that connects to UCM must register

with UCM. After registration, a system’s privileges and accessibility can be

defined1. Query2. Insert3. Update4. Delete

Defines grant access to Siebel UCM Privileges are granted at the BO layer

13


How to register an external application with UCM

Use the system registration view1. Select Administration –Universal Customer Master > System

registrations2. Click New3. Enter a system name, id and protocol type(HTTP,MQSeries)4. Below System registrations applet Select New and assign BO

privileges in theSystem Privileges.

14


Registering an External Application

15


External System referencing: Cross-Referencing

Cross-referencing stores source identification information for data in the UCM UCM stores identification information about external systems

that have been registered. Cross-referencing provides a one-to-many mapping of customer

records acrossmultiple systems

Allows UCM to keep track of the same customer record across multiple systems

A single UCM record maps to one or many records in any registered application

UCM supports cross-referencing for the following objects:1. Contacts2. Accounts3. Households

16


Cross-Referencing Cross-reference entries are created automatically with the creation

of every new account, contact, or household record External systems send requests to insert a record

If there is an existing cross-reference entry, the entry is updated

If there is no existing cross-reference entry, a new record is inserted

The system ID is the value assigned when the system was registered and system name is the associated name There are multiple possible external Ids to account for

situations where multiple records in an operational application map to a single record in UCM Only the first is stored in the ID tag in a message request,

by default

17


External System Referencing: Universal Unique ID (UUID)

Another mechanism to track the same record across multiple applications Instead of keeping track of the system IDs., UCM publishes a

UUID to the operational application

The UUID is not exposed in the user interface Is stored in a column of the S_PARTY table Can be exposed using Siebel Tools

Many customers use cross-referencing and the UUID at the same time

18


Data Quality Options Data quality consists of

1. Data Cleansing2. Data Matching(De-Duplication)

Data Cleansing1. Address validation and standardization2. Capitalization3. Abbreviations

Data Matching identifies possible duplicate records for account,contact and prospect records

Identify duplicates and merge them based on certain key attributes.

E.g. from our previous case, if we decide that the key attributes of a unique customer are : first and last names and say the passport number. So, assuming we have the passport number stored in both the .NET and J2EE application, data matching in UCM will be able to identify that customer # 1-123 and AKS123 are one and the same.

19


How to configure Data Quality To configure data quality :

1. Enable the data quality component group2. Set data quality parameters3. Specify data quality settings4. Specify data quality settings: Site Map > Administration

—Data quality > Data quality settings During integration cycle disable deduplication to simplify the

environment.

20


What to Do with Duplicate Records Once suspect duplicate have been identified, the records can be

evaluatedand: Merged with existing records Introduced as new UCM records (promoted)

For records that are merged, UCM: Stores the duplicate record, with Merged as the type, and the

surviving record, with History as the type, in the Source Data History table

Merges parent records using survivorship rules to generate a new merged surviving parent record

Updates the parent surviving record with the new merged surviving record

Deletes the duplicate record

21


Survivorship Rules Customer data originates in multiple systems Conflicts arise when different sources have different data for the

same attributeFor eg: two systems have different addresses for customer

The role of UCM is to resolve these conflicts, which ensures that the best version record survives

Survivorship rules are an automated mechanism for consolidating customer data to produce a best-version record Evaluates updates from each registered system Applies rules to determine which is the best value for each

attribute Can be constructed by a system integrator or a CDI data

steward

Three comparison methods Recent History Source

22


Survivorship Rules and Match Scores UCM allows two thresholds to be set depending on the match

scores from the matching process

High Score - 90

Low Score - 70

Records with high match scores can be auto-merged, applying survivorship rules

Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution

Records with low match scores will be left as separate records

23


Setting Match Thresholds Auto-threshold: Determines the high matching score threshold

Account Auto Threshold Contact Auto Threshold

Manual threshold: Determines the low matching score threshold Account Manual Threshold Contact Manual Threshold Can be constructed by system integrator or CDI data steward

Can be set by a system integrator or a CDI steward

24


Survivorship Rule Structure Employ comparison criteria to determine if a new value for a given

attribute coming from any source is better than the existing value in the best-version record Three comparison methods:

Recent History Source

Multiple rules can be constructed for each object (account or contact) but only one rule can be active at a time

Use attribute groups A group of fields that survive or get overwritten together Makes survivorship rule creation less cumbersome

Each field within an object can only be included in a single attribute group To avoid survivorship rule conflicts

25


Building Survivorship Rule Three primary steps:

Create an attribute group for the survivorship rule Create the survivorship rule logic Activate the survivorship rule

26


Setting Survivorship Rules

27


Unmerge For records that have been merged

Surviving record--- record that remains after the merge Victim record ----- record that is merged and ceases to exist

If merged records are unmerged after changes have been made to be the merged (surviving) record All changes and updates since the merge will be applied to the

surviving record The victim record will be resurrected from the Source Data

history tableA

B

A1

A2

BA2

MergeUnmerge

Changed to merged record

28


Outbound Data Flow: Publish and Subscribe

Publish and subscribe allows records inserted or updated in UCM to be published to external systems Publication can be targeted to any system registered with UCM

The mechanism for publication is a Siebel Workflow Workflow policies track changes to UCM records and flag them

for production Then Workflow processes collect the appropriate information

and guarantee publication of changes to specified (subscribed) systems

Publish and subscribe requires that publication parameters be specified Can be real-time or daily-batch

29


Module 2: Overview Of Application Integration

30


Application Integration and the UCM SDH tables

UCM integration extends the EAI toolset by transferring data to SDH tables prior to placing data into Siebel base tables

SDH tables provide a queuing station for processing incoming records

Records can be analyzed for future processing Data must be processed at this layer before entering standard

Siebel base tables

This is the source of integration process differences between UCM and

the standard Siebel CRM Enterprise product

31


Data Transfers to SDH tables Data transfers to SDH tables use traditional methods

Data layer data loads via the EIM Business layer data loads via standard integration application

procedures Data transfers paths to SDH tables analogous to standard CRM

case No data cleansing or matching

SDH Tables Base Tables

Business Layer Data Loads

Data Layer Data Loads

32


Data Transfers to Siebel Base Tables Are independent of how data is placed on SDH tables

Data layer Business logic layer

During this transfer(insert) data quality is activated Cleansing and matching Application of survivorship rules

Implemented as a UCM workflow process– The UCM Batch Manager

33


UCM Service Requests External Systems request services from a UCM application

Insert Query Update Delete

UCM must determine which external system is requesting the service To validate privileges

The System ID must be supplied in addition to the traditional inputs even if the UCM application is not physically connected to an external operational system

34


UCM External System Parameters Request for service can only be accepted from a known

source Registered system

System ID System Name

The records to be operated on also have UCM specific information associated with them Unique external record ID Unique user ID Transfer code describing data transfer mechanism

35


Inbound Data flow for the business layer

Request for UCM application services Submitted through a (CRMML) input file Security service checks privileges of requesting external

system Transaction service executes operations

Siebel AdapterThrough

Dispatch Service

Security and Transaction Services

Input data (CRMML)

Outbound Data

To SDH tables

36


CRMML Input Files Contain

Information about external system --- system ID Service being requested Methods and arguments to be used

User does not have to build the integration objects Internal integration objects provided by the UCM application The CRMML file directs the UCM application on which the

integration object to use

User does not have to specify UCM specific record information The UCM application will make these assignments when placing

records on SDH tables

37


Security and Transaction Services The security service checks the system ID specified in the input

(CRMML) file for Insert ,Update, Query and Delete privileges If the system is not registered data will not be processed

Transaction manager will perform the requested operations Uses an internal UCM integration object to obtain the data

necessary to process a request

38


Implementing Inbound Data Flow at the Business Layer

Inbound Requests are processed by a workflow process HTTP Queue Based File Transport

Protocol specified when system is registered

Siebel Adapter Through Dispatch

Service

Security and Transaction Services

Input Data

(CRMML)Outbound Data

To SDH tables

Server Party Package

39


Inbound/Outbound Processing UCM has automated much of the inbound processing --- a user

must:Prepare input(CRMML) file

Run (appropriate) Server Party Package workflow to place records of SDH tables

Run UCM Batch workflow to activate data quality and place data on Siebel base tables

The UCM application supports an outbound publish and subscribe facility

Insert and Update records can be published to external system subscribers

Individually or as a groupIn batch or real time mode

40


Module 3: Initial Data Loads into UCM

41


Populating UCM Data Use business layer imports to enter , update or delete small

amounts of data in the base tables Use the EIM server task to populate the SDH tables ( through

interface tables)

Use the UCM batch manager to transfer data from SDH tables to Siebel Base tables

Cleanse and matchNon-Siebel loading tool

External data – may be

cleansed

UCM Database

Interface table

UCM SDH tables

Siebel Base tables

42


Staging Tables The UCM application uses two sets of staging tables before making

data available to the Business Logic layer through Siebel Base tables

The EIM staging tables: Data is staged to be transformed into a Siebel standard format

The Source ,Data and history tables: Data is staged for cleansing and matching

43


Data Quality Options for Initial Loads Data may be cleansed and matched while being transformed from

the source , data and history tables or preprocessed before loading into EIMtables

Cleansing and matching during data import adds loading latency ----time overhead

If the Siebel Universal Connector is being used for cleansing and/or matching , it will need to be integrated into the application

44


Siebel Base Tables User data is stored in one or more Siebel base tables Relationships between base tables (referential integrity) rely upon

primary and foreign keys based on ROW_IDs ROW_ID is the system primary key for every base table ROW_ID is a system-generated value

PK

S_BU

PK

S_ORG_TXT

FK FK

S_ADDR_ORG

45


User Keys Based on multiple columns, user keys are used to uniquely identify

a row for EIM Primary and foreign keys based on ROW_ID are used for system-

wide referential integrity

ROW_ID Name LOC BU_ID

PK FKUser Key

S_UCM_BUS_ORG_EXT

46


UCM Data Loads Loading Data into the EIM Tables Loading Data from the EIM Table to the SDH Tables Loading Data from the SDH Table to the Base Tables

47


Interface Tables Staging area for data to be imported, updated, or merged into

Source Data and History (SDH) Staging area for data imported by EIM EIM tables do not have ROW_Ids assigned

External Data

External Database

EIM_UCM_ORG

Interface Table

S_UCM_ORG_EXT

S_UCM_CONTACT

SDH Tables

48


Populate Interface Tables Populate source data into interface tables using a non-Siebel

process Use database vendor utilities to load data from flat files Use SELECT INTO to pull data from other SQL databases Write programs or stored procedures to cleanse or preprocess

and insert rows Use direct SQL insert, update, or delete statements to update

columns in EIM tablesExternal Data

Non-Siebel process

EIM_UCM_ORG

49


Required Interface Table Columns In addition to standard non-nullable columns

IF_ROW_BATCH_NUM ---- Batch number IF_ROW_STAT --- Status ROW_ID – Row ID

The UCM application also requires a non-null inputs UCM_UID – A unique UCM record identifier UCM_TYPE_CD --- SDH type (example : batch for initial loads) EXT_SYSTEM_NUM – External System ID UCM_EXT_ID --- A unique external record identifier (usually

assigned by the external system)

50


Source History Data (SDH) and EIM Schema

UCM Source Data History and EIM Tables

S_UCM_ORG_EXT Source and History table for accountsS_UCM_CONTACT Source and History table for contactsS_UCM_ORG_CHLD Source and History table for child objects

of accountS_UCM_CON_CHLD Source and History table for child objects

of contactEIM_UCM_ORG EIM interface table for S_UCM_ORG_EXTEIM_UCM_CON EIM interface table for S_UCM_CONTACT

EIM_UCM_ORG_CHLD EIM interface table for S_UCM_ORG_CHLDEIM_UCM_CON_CHLD EIM interface table for S_UCM_CON_CHLD

51


Referential Integrity User key column mappings in interface tables are used to resolve

ROW_Ids for SDH tables, maintaining referential integrity

User key

1-8DFK(ROW_ID)

PK(ROW_ID)

1-8D

Interface Table

UCM SDH Tables

Acme

52


Enterprise Integration Manager (EIM) Server task that manages the exchange of data between Interface

tables and Source Data History tables

EIM_UCM_ORG

External Data

S_UCM_ORG_EXT

S_UCM_CONTACT

53


Running the Enterprise Integration Manager

The EIM server task is run as a workflow from Administration – Server > Jobs

54


Run as EIM Task with Parameter Settings

Creates an operating system (OS) process that writes log entries, reads an .ifb file, loads a dictionary and executes EIM steps

55


Configuration (.ifb) file Role for initial data loads

Specifies the batch , order and type of processing desired Should be placed in c:\Susrvr\siebsrvr\ADMIN

For imports [Siebel Interface Manager] PROCESS = IMPORT EIM_UCM_CON [IMPORT EIM_UCM_CON] TYPE = IMPORT TABLE = EIM_UCM_CON BATCH = 100 ONLY BASE TABLE = S_UCM_CONTACT

56


Use Log Information to Troubleshoot Navigate to Administration – Server > Tasks

Error information in Task view or log file

57


Verify Data Load into SDH Tables Navigate to Administration – Universal Customer Master > Contact

(Account) > Source Data and inspect the Source Data table

58


UCM Batch Manager Transfers data from SDH tables to Siebel Base tables Implemented by separate workflows for Accounts and Contacts

UCM Account Batch Data Management workflow UCM Contact Batch Data Management workflow

S_UCM_ORG_EXT

SDH table

SDH table

Base column

Base Table

S_ORG_EXT

59


Running the UCM Batch Manager The UCM Batch Manager is run as a workflow from Administration-

Server > Jobs

60


Verify the Data Load into Siebel Base Tables

Navigate to Site Map > Contacts (Companies) > Contacts(Accounts) List and inspect the list

61


Determining Data Mapping Need to analyze and determine :

Which Siebel base table columns will store external source data Which interface (EIM) tables and Source Data History (SDH) tables will be used to import from source to destination

Source (External

How to get there

Interface(EIM)

SDH

Destination(base)

Where data ends up

62


How to Get There After verifying and documenting the source-to-base data mappings,

determine EIM-to-SDH and SDH-base mappingsSource Entity

Source Attribute

EIM Table and Column

SDH Table and Column

Base Table Base Column

Customer Name S_ORG_EXT NAME

Customer Phone S_ORG_EXT MAIN_PH_NUM

Address S_ADDR_ORG ADDR

S_ADDR_ORG CITY

S_ADDR_ORG ZIPCODE

Interface(EIM) SDH

(UCM)

63


Table Column Mappings Can be determined using Siebel Tools

EIM Interface tables Source Data History Tables Source Data History Tables Base tables

64


Module 4: The UCM Batch Workflow and SDH Tables

65


The UCM Batch Manager and SDH Tables

This module will discuss two aspects of UCM Batch Manager operations on SDH tables. Data transfer to Siebel Base Tables Data transfer for child objects

66


The UCM Batch Manager Batch processes insert requests queued on the Source, Data ,and

History (SDH) tables Initial data loads (legacy systems) Incremental inserts

Moves records from SDH tables to Siebel base tables Example: S_UCM_CONTACT S_CONTACT

Two flavors of batch workflows for accounts and contacts UCM Account Batch Data Management Workflow UCM Contact Batch Data Management Workflow

67


UCM Account Batch Data Management Workflow

Start

Query by Id

CDM Cleanse

CDM Match

Get Error Code

Match id?

Error? ProcessNoMatch

ProcessAutoMatch

ProcessReview

Error? Converter

PubSub

End

68


UCM Batch Workflow Steps Query By Id CDM Cleanse CDM Match Process NoMatch Process AutoMatch Process Review

69


Query By ID Invokes the Query method of the EAI Siebel Adapter business

service Reads an object to be inserted from a SDH table Outputs a Siebel message

Query By Id

OutputintObjectName

PrimaryRowId

SiebeMessage

Workflow Process Props

SDH int Obj Name

Object Id

Siebel Message

70


CDM Cleanse Invokes the Cleanse method of the UCM Data Quality Manager

business service Reads a Siebel Message containing object to be inserted from

SDH table Outputs a cleansed XML representation of the original Siebel

Message

CDM Cleanse

IntObjectName

SiebeMessage

XML Hierarchy


Base int Obj Name

Siebel Message

Cleansed Prop

71


CDM Match Invokes the Match method of the UCM Data Quality Manager

business service Reads an XML representation of a Siebel Message containing an

object to be inserted from SDH table Output indicates if record is a duplicate and the auto-merge

level

CDM Match

XML Hierarchy

NumAutoMatch

MatchingrowId


Cleansed Prop

Number of AutoMatch

Cleansed Prop

72


To Merge or Not to Merge The workflow splits the output of the match process into three

cases (CaseType) No match ---- insert record Potential match --- activate survivorship rule to determine if

merge is appropriate Transfer record for manual resolution

73


Process NoMatch Is activated in the event that the incoming record is not scored as a

match Record is inserted Invokes the HandleMatchCase method of the UCM Batch

Manager business service

ObjectType

XML HierarchyProcess NoMatch

CaseType

TurnOnSE

XML Hierarchy


Matching Id

Turn On Survivorship

Processed Prop

Object Type

Cleansed Prop

74


Process AutoMatch Is activated in the event that the incoming record is scored as a

match Record evaluated by survivorship rule --- is either merged or

inserted Invokes the HandleMatchCase method of the UCM Batch


ObjectType

XML Hierarchy

Process AutoMatch

MatchingRowId

TurnOnSE

XML Hierarchy


Matching Id

Turn On Survivorship

Processed Prop

Object Type

Cleansed Prop

CaseType

75


Process Review Is activated in the event that the incoming record is scored as a

match but not resolved by survivorship rule Record is posted in Duplicate (Accounts) view Invokes the HandleMatchCase method of the UCM Batch


ObjectType

XML Hierarchy Process Review

CaseType

XML Hierarchy


Processed Prop

Object Type

Cleansed Prop

76


Post Process Activities In the event that the record is resolved through an insert or a

merge A real-time message can be published

In the event record is queued for manual resolution The workflow is terminated and no information is published(in

real--time mode)

77


Address for Contacts Contact business objects come preconfigured with address child

busniess components

78


The UCM Data Model and Child components

All UCM child components for each business object (Account and Contact) are stored in one table The explicit component object type is configured in each UCM

business component to distinguish the child components

Field

SDH

Column

BC BO

SDH

table

Busniess Layer

Data Layer

79


Address for Contacts Contact business objects come preconfigured with address child

business components

UCM Contact SDH

UCM Contact Address SDH

S_UCM_CONTACT

S_UCM_CON_CHILD

Primary Business

Component

Business Component

Table

UCM Contact SDH

Business Object

80


Source Data and History : Data schema

Provide the infrastructure necessary to support child structures Example : addresses for contacts

S_ORG_EXT

S_UCM_ORG_EXT

S_UCM_ORG_CHLD

S_UCM_CONTACT

S_UCM_CON_CHLD

S_CIF_EXT_SYST

S_CONTACT

Source data History Table

Base table

Source Data History Child table

External System information table

81


Creating UCM SHD Child Business Components

The UCM data model can easily be extended to include custom SDH components

82


Example: Address for Contacts The UCM SDH Contact Address business component is UCM Contact

Address Source Data and History

83


Custom Child Business Components Name Business Components and Tables

Assign Data Fields

Configure UCM Fields

Configure UCM Batch Workflow

84


Name Business Components and Tables

Copy the Siebel base business component and rename it Siebel suggested standard child business component names: “UCM” + <Parent BC Name> + <Child BC Name> + “SDH”

Copy of UCM Contact Address Source Data and History

85


Assign Data Fields Change the predefined column names (‘ATTRIB_N’) for the business

component to the UCM SDH table Name field

86


Configure UCM Fields There are five UCM specific fields that must be configured

Component Field Name

Column Predefault Value

Object Name OBJECT_NAME Party Object name(Account/Contact)

Object Table OBJECT_TABLE UCM Party base table

Object Type OBJECT_TYPE Equal to child name

Par Rpw Id PAR_ROW_ID Parent Row IdUCM UId UCM_UID Field identifier

87


Configure UCM Batch Workflow The OBJECT_TYPE is used for the search specification in the UCM

Batch Workflow parameter list Search Specification = Object Type

Example for Addresses: [Object Type] = ‘address’

88


Module 5: UCM Messaging and CRMML

89


CRMML An XML based markup language for data exchange and remote

business process invocation Used for communication with UCM

Inbound requests (insert, query, update, delete) Responses Publishing messages

Back-OfficeApplication UCM

Request

Response

Publication

These exchanges use CRMML messages

90


Example: Insert Contact Profile <CRMML> <CRMHeader> <FromSystem> <SystemInfo> <SystemId>System123</SystemId> <SystemName>External App</SystemName> </SystemInfo> </FromSystem> <ToSystem> … </ToSystem> <MessageDescriptor> … </ MessageDescriptor> </CRMHeader> <CIFPartyPackage> <InsertContactProfile> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> …. </Contact> <InsertContactProfile> </CIFPartyPackage> </CRMML>

Requested transaction,or ‘method’

Method “argument”

‘System’ requesting the info

91


CRMML and Integration Objects Integration Objects

Define the structure of the CRMML message Are used as templates to create memory-resident instances

of data for processing by business services UCM provides integration objects for different parts of the

CRMML messageIntegration Object

TypeDescription

Envelope Integration Object

Represents CRMML header structure

Dispatcher map integration object

Contains rule sets for mapping from method to integration objects

UCM integration objects Represent the method argument (Contact, Account, etc)

92


CRMML Header Contains

Information identifying sending and receiving systems Used by Universal Customer Master for security

verification Message descriptor identifying the message

<CRMHeader> <FromSystem> <SystemInfo> <SystemId>System123</SystemId> <SystemName>External

App</SystemName> </SystemInfo> </FromSystem> <ToSystem> … </ToSystem> <MessageDescriptor> … </ MessageDescriptor> </CRMHeader>

93


CRMML Header : System Info Used to describe both the sending and receiving systems

<FromSystem> - sender <ToSystem> - receiver

For inbound messages, FromSystem SystemId is matched against registered systems as part of UCM’s security check

<Systeminfo> <SystemId>123-456-789</SystemId> <SystemRole>Insurance App</SystemRole> <SystemName>Client System</SystemName></Systeminfo>

SystemId used for security

verification

94


CRMML Header : Message Descriptor Identifies the message

MessageId : Unique ID for the message Timestamp: when message was issued TimeToLive: how long (after timestamp) message is valid TransactionScope : reserved for future use

<MessageDescriptor> <MessageId>1T3D2005083115:45:32</MessageId> <TransactionScope>any</TransactionScope> <Timestamp>Sunday</Timestamp> <TimeToLive>OneYear</TimeToLive></MessageDescriptor>

95


CRMML Header : Summary CRMHeader element has the following structure

<CRMML>

<CRMMLHeader>

<Body>

<FromSystem>

<ToSystem>

<MessageDescriptor>

<SystemInfo>

<SystemInfo>

96


CRMML Header: Integration Object The CIF Envelope Integration Object is used to process the CRMML

Header

97


CRMML Body Contains one or more message elements representing:

Inbound requests from an external applications to insert, query, update, or delete a database object

Outbound response to a requested transaction Outbound publication messages to be sent to an

external subscribing application Each message element includes:

Software package – must be CIFPartyPackage Method --- the requested action Argument – the argument to the action

<SoftwarePackage> <Method> <Argument>

<CIFPartyPackage> <InsertContact> <Contact>

98


Mapping to Integration Objects UCM methods and arguments are mapped to integration objects

according to a set of mapping rules These mapping rules define:

What methods are allowed What arguments those methods take

<CIFPartyPackage> <InsertContact> <Contact> <LastName> </LastName> <FirstName> </FirstName>

…….

CIFContact

Contact

FirstNameLastName

Methods contain Arguments,which contain Fields

Integration Objects contain Integration Components, which contain Integration Component Fields

99


Mapping to Integration Objects Continued

The mapping rules are defined in user properties of a mapping integration object The default mapping object is CIFDispMap

CRMML/CIFPartyPackage/InsertContact -> CRMML/CIFPartyPackage/InsertContact;a;b;CIFContactInsert;CIFContactInsertRs; IXML_Operation_ADD

Envelope Package Method

Integration Object for inbound

requests

Integration Object for outbound responses

100


CRMML Structure: Summary Structure of a base CRMML message

<CRMML>

<CRMHeader>

<CIFPartyPackage>

<FromSystem>

<ToSystem>

<MessageDescriptor>

<SystemInfo>

<SystemInfo>

<InsertContact> <Contact>

Package Method Argument

101


Complex CRMML Messages Arguments may contain sub-arguments

For example: Contacts contain addresses These sub-arguments are represented by sub-elements in the

CRMML message

Method<InsertContactAddress>

Argument<Contact>

Method<InsertContactAddress>

<Contact_INSPersonalAddress>



102


Example: Insert Contact Address <CRMML> <CRMHeader> ….</CRMHeader> <CIFPartyPackage> <InsertContactAddress> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> <ListOfContact_INSPersonalAddress> <Contact_INSPersonalAddress IsPrimaryMVG=“Y”> <INSPersonalStreetAddress>100 Main St.</ INSPersonalStreetAddress> <INSPersonalCity>York</INSPersonalCity> <INSPersonalState>AL</INSPersonalState> <INSPersonalPostalCode>78221</INSPersonalPostalCode> <INSPersonalCountry>USA</INSPersonalCountry> </Contact_INSPersonalAddress> <Contact_INSPersonalAddress IsPrimaryMVG=“N”> <INSPersonalStreetAddress>50 Center St.</INSPersonalStreetAddress>

……. </ListOfContact_INSPersonalAddress> </Contact> </InsertContactAddress> </CIFPartyPackage> </CRMML>

103


Module 6: UCM Server Party Package Workflows

104


Party Package Workflows Handle incoming requests

Check source system privileges Create memory-resident instances of data for processing by

business services, based on integration object definitions Execute requested operation Return response to requesting system Publish (real-time) to subscribing systems

105


Party Package Workflows continued

Include

Workflow Process Name DescriptionUCM Server Party Package Workflow (File Transport)

Reads CRMML message from a file and writes response to a file (for testing)

UCM Server Party Package Workflow (MQSeries Server Transport)

Used for HTTP and MQSeries message transports.

UCM Server Party Package Workflow(MQSeries Server Transport Error Handling)

Also for HTTP and MQSeries transports, Models error handling.

106


UCM Server Party Package Workflow (File Transport)

Workflow is composed of the following steps:

Start Read From File

Convert to XML Hierarchy

Dispatch Message

SecurityCheck

Convert Request In

TransactionManager

Convert Request

Out

RealtimePub/Sub

Convert to Hierarchy

PubSub

Write To File

End

107


Read From File Invokes the Receive method of the EAI File Transport business

service Reads contents of a file into string Input file should contain a valid CRMML message

WorkFlow Process Props

Input File Name

XML

Read From File: EAI File Transport Receive

File name

<value>

Start


108


Convert to XML Hierarchy Invokes the XMLDocToXMLHier method of the XML Hierarchy

Converter business service Converts an XML string into an XML hierarchy object


XML

Core Req Prop

Convert to XML Hierarchy: XML Hierarchy Converter XMLDocToXMLHier

<value>

XML Hierarchy

Dispatch message

Read From File

109


Dispatch Message Invokes the DispatchMessage method of the UCM Dispatcher

business service Verifies structure of CRMML message Acts information to message components that is used by

downstream business servicesWorkFlow Process Props

Core Req Prop

Disp Req Prop

Dispatch Message: UCM DispatcherDispatchMessage

XML Hierarchy

Security Check

XML Hierarchy


110


Dispatch Message Continued

Uses two integration objects, specified as user properties of the business service, to represent the message

User Property Name

Value

DispatcherMapName

CIFDispMap

XMLEnvIntObjectName

CIF Envelope

Integration object used to process body

Integration object used to process header

111


CIF Envelope Integration Object Defines the allowed structure of the CRMML message header.

112


Dispatcher Map Integration Object Contains a set of rules to map methods to integration objects used

to process those methods Each rule is a user property of the mapping integration object

Name identifies the message content that must be matched Value specifies the information that should be added to the

method when a mapping is found

CRMML/CIFPartyPackage/InsertContact -> CRMML/CIFPartyPackage/InsertContact; a;b;CIFContactInsert;CIFContactInsertRs;IXML_Operation_ADD

Integration Object for inbound

requests

Integration Object for outbound responses

Operation for the business process

Reserved for future use

Location where information gets

inserted

Content to match in order to trigger this

rule

113


Dispatch Message Continued

Sample message before Dispatcher additions

<CRMML> <CRMHeader> ……. </CRMHeader> <CIFPartyPackage> <InsertContactProfile> <Contact> …. </Contact> </InsertContactProfile> </CIFPartyPackage> </CRMML

114


Dispatch Message Continued Sample message after Dispatcher additions

<CRMML SiebelFINSObjType=“Envelope”> <CRMHeader SiebelFINSObjType=“Header”> ….. </CRMHeader>

<CIFPartyPackage SiebelFINSObjType=“Body”> <InsertContactProfile SiebelFINSOperation=“IXMLOperation_ADD” SiebelFINSIntObjName=“CIFContactProfileInsert”

SiebelFINSRespIntObjName=“CIFContactProfileInsertRs” SiebelFINSMapOut=“b” SiebelFINSMapin=“a”> <Contact> ….. </Contact> </InsertContactProfile> </CIFPartyPackage> <CRMML>

Content added here

Additional content added by Dispatcher

Dispatcher adds these attribute values which come from the triggered mapping rule

115


Security Check Invokes the SecurityCheck method of the UCM Security

Manager business service Verify the SystemId in the message header against system

registrations Return an error if the system is not recognized

For each method in the request, verify that the system has the privilege required to execute the operation Failed operations are removed from the request and

put into the ‘Fail Security Req Prop’ property set


Disp Req Prop

Fail Security Req Prop

Security Req Prop

Security Check: UCM Security Manager Security Check

XML Hierarchy

Security Check

XML Hierarchy

Dispatch Message

FailSecurityHierarchy

116


Security Check Continued

User properties map operation (inserted by the Dispatcher) to privileges (assigned to systems in system registration)

Operation Name PrivilegeIXMLOperation_ADD InsertIXMLOperation_DELETE DeleteIXMLOperation_QUERY QueryIXMLOperation_UPDATE Update

In order to perform this operation Requesting system must be

registered with this privilege

117


Convert Request In Invokes the XMLPropSetToPropSet method of the UCM Converter

business service Each method is converted into the format required by the

Transaction Manager Requesting System information is extracted from the header to

make it available to downstream business services Uses integration object, CIF EnvelopeWorkFlow Process

Props

Security Req Prop

CIF System prop

Tran Req Prop

Convert Request In: UCM ConverterXMLPropSetToPropSet

XML Hierarchy

Transaction Manager

XML Hierarchy

Security Check

CIFSystemContainer

118


Convert Request In Continued

Sample converted method

<CIFPartyPackage….> <InsertContactProfile….> <Contact> <FirstName>Test</FirstName> <LastName>Contact</LastName> <Status>Active</Status> ….. </Contact> </InsertContactProfile></CIFPartyPackage>

<CIFPartyPackage….> <ListOfCIFContactProfileInsert….> <Contact> First_spcName=“Test” Last_spcName=“Contact” Status=“Active” ….. /> </ListOfCIFContactProfileInsert></CIFPartyPackage>

Before After

119


Transaction Manager Invokes the Execute method of the UCM Transaction Manager

business service Performs operations in message by invoking services

configured in its user properties

TurnOnSE


Tran Req PropTurn On Data Mgmt Turn On Survivorship Status Object

Notification Req Prop

Trans Resp Prop

Transaction Manager:UCM Transaction Manager Execute

XML Hierarchy

Convert Request Out

XML Hierarchy

Convert Request In

TurnOnDM

StatusObject NotificationHierarchy

120


Transaction Manager Operations Supported Operations

Operation DescriptionInsert Insert into Source Data history tables with

status,Queued.Update Insert into Source Data History tables with

status,Update. Invoke Survivorship and Data Quality Manager, if enabled.Insert updated record directly into base tables.

Query Query base tables.Delete Locate record to delete based on user key

in message.Store delete request in Source Data History table.Store historical best version.Delete record.

121


Transaction Manager Configuration Configuration Transaction Manager inputs in the workflow to

Enable data management and survivorship TurnOnDM– enable data management (default is false) TurnOnSE--- enable survivorship (default is false)

Include a representation of the affected object in the response to the requesting system StatusObject (default is true)

122


Convert Request Out Invokes the PropSetToXMLPropSet method of the UCM Converter

business service Generates a response that combines information from its inputs Response will be converted to an XML hierarchy and then sent

back to the requesting systemWorkFlow Process Props

Tran Resp PropNotification Req Prop CIF System Prop

Fail Securtiy Req Prop

Conv Resp Prop

Transaction Manager:UCM Transaction Manager Execute

XML Hierarchy

XML Hierarchy

Transaction Manager

CIFSystemContainer

NotificationHierarchy

FailSecurityHierarchy

RealtimePub/Sub

123


Considerations Error Handling Configuring Inbound Server Communication

124


Error Handling The UCM Server Party Package Workflow (MQSeries Server

Transport Error Handling) demonstrates how error handling could be added to a workflow.


Dispatch Message

Security Check

Convert Request In

Transaction Manager

Error Handler:UCM Converter:

GenerateErrorPropSet

Error Handler:UCM Converter:

GenerateErrorPropSet

End

125


Error Handling Continued

The UCM Converter service includes a method to aid with error handling


Error Code Error Message Disp Req Prop

Error Hierarchy Prop

UCM Converter:GenerateErrorPropSet

XML Hierarchy

ErrorCode ErrorMessageText

XML Hierarchy

126


Configuring Inbound Server Communication

Your UCM environment must be configured for inbound server communication

Configuration procedure depends on the system transport protocol MQSeries Transport HTTP Transport

127


Module 7: UCM Batch Data Management Workflows

128


Batch Data Management Workflows Process data queued in Source Data and History tables

During initial data load As a result of an insert request processed by Transaction

Manager Separate workflows for contacts and accounts

UCM Contact Batch Data Management Workflow UCM Account Batch Data Management Workflow

Each workflow performs data cleansing, matching, and real-time publication (if enabled)

129


UCM Contact Batch Data Management Workflow

Performs the following actions on a single input record Cleansing Matching Real-Time publication

Start

Error match

Query by ID

CDM Cleanse

CDM Match Auto Match?

Get Error Code

Error? Process NoMatch

Process AutoMatch

Process Review

RealtimePub/Sub

Converter

PubSubEnd

Matching

Publishing

Yes No

True

False

130


Data Cleansing Performed by Cleanse method of the UCM Data Quality Manager

business service Invokes third-party data cleansing software (if enabled) Output is cleansed record

Start

Query by ID

CDM Cleanse

Retrieve record data for processing

Perform data cleansing

131


Data Matching (Deduplication) Performed by Match method of the UCM Data Quality Manager

business service Three possible outcomes:

No match found Match found that must be reviewed Match found that can be processed automatically

CDM Match Auto Match?

Get Error Code

Error? Process NoMatch

Process AutoMatch

Process Review

Error match

Yes No

132


Data Matching Continued

All three match cases are processed by the same business service method:

UCM Batch Manager HandleMatchCase

Process NoMatch

Process AutoMatch

Process Review

133


Data Matching Continued

Thresshold settings determine how match is handled

Process NoMatch

Process AutoMatch

Process Review

High Score - 90

Low Score - 70

Records with high match scores can be auto-merged,applying survivorship rules.

Records with match scores in the middle will be brought to the attention of the data steward for manual review and resolution.

Records with low match scores will be left as seperate records.

134


Adjusting Threshold Settings Configure user properties of the UCM Batch Manager business

service to adjust merging thresholds

70Contact Manual Threshold

90Contact Auto Threshold

70Account Manual Threshold

90Account Auto Threshold

Default ValueUser Property

135


Publishing Data changes can be published to subscribing systems

Result of auto merging of records Result of insert of record (no match condition)

Publishing is addressed in the next moduleProcess NoMatch

Process AutoMatch

RealtimePub/Sub

Converter

PubSub

End

True

False

Convert data to format for publishing

Publish Transaction

136


Running Batch Data Management Workflows

Create a Job

Configure Job Parameters

Run the Job

137


1. Create a Job In the Administration – Server Management > Jobs view. Create a

new job to run the Workflow Process Batch Manager component/job

If job is repeating specify repeat info in

job details

Select this component/job

138


2. Configure Job Parameters Include

Search Specification: Identifies items to process Workflow Process Name : Identifies workflow to invoke on each

item

139


3. Run the Job Once the job parameters are set, run job

Click start button to start the job

140


Module 8: Publish and Subscribe

141


Publish and Subscribe Allows records inserted or updated in UCM to be published to

external systems Distributes the best version customer profile Systems registered as subscribers receive publication

messages

UCM

Subscriber 1

Subscriber 2

Subscriber 3

Update: Contact123

Update: Contact123

Update: Contact123

142


Publish/Subscribe Modes Publish and Subscribe framework supports two operative modes:

Real-time Transaction are published as they happen Each individual transaction is published to real-time

subscribers Daily Batch Mode

Transactions are published once per day Best version of record is published, individual transactions

are not

Systems can mix and match modes Contact updates received in real-time Account updates received from daily batch job

143


Publication Messages Publication messages are CRMML messages

FromSystem is UCM ToSystem is subscriber

Message body content is constructed using integration objects Real-time publishing uses the response integration object

associated with the particular UCM transaction Batch publishing uses the integration object type specified as

input to the both Publish/Subscribe business service

144


Workflows and Real-Time Publishing UCM Workflows include steps to perform near real-time publishing

Decision step invokes publishing method if publishing is enabled

PubSub

Convert to Hierarchy

RealtimePub/S

ub

PubSub

Convert Request

Out

Process AutoMatch

RealtimePub/S

ub

PubSub

Converter

End

Transaction Manager response is published

Process NoMatch

Result of match is published

UCM Server Party Package Workflow UCM Contact/Account Batch Data Management Workflow

145


Real-Time Publishing of Insert Operations

Insert operations are processed in two steps: Queue insert in SDH tables Process insert, performing any necessary data cleansing and

matching Both steps will generate a publish message The subscriber system should only process the second message

Subscriber should ignore publish messages reporting that an insert operation has been queued.

146


Workflows and Batch Publishing UCM Daily Publish workflow supports regular batch publishing

Start UCM Daily Publish End

147


UCM Publish/Subscribe Business Service

Workflows invoke this business service to perform publishing Method

PublishMethod Inputs

XML Hierarchy– Contains data to be published

Integration Object– Used to generate the publication message (batch

publishing only)

148


Configuring Publishing: Real-Time Set workflow user property to enable publishing

User Property names differ! Set to true to enable publishing

149


Configuring Publishing: Real-Time Continued

User Property names differ! Set to true to enable publishing

150


Configuring Publishing: Daily Batch Configure a job to run the UCM Batch Publish Subscribe component

This job will run the Daily Batch workflow

Integration object to use

Number of records to processed in one

batch

151


Subscribers Registered systems can be configured as subscribers to receive

publication messages in real-time or batchURL where publication

messages are sent

Required for batch publishing

Updated each time a message is published to this subscriber

152


Module 9: Wrap-Up

153


Course Goal To enable participants to integrate Siebel Universal Customer

Master (UCM) with external applications Legacy systems External databases

Was achieved by covering the following topics Initial data loads into UCM at the data layer Incremental data loads into UCM at the business logic layer Managing outbound data flows via publish and subscribe The CRMML messaging protocol for requesting UCM services

154


Initial Data Loads into UCM at the Data Layer

Batched data layer loads into UCM are a two-stage process Loading from Siebel Interface table to Source Data and History

(SDH) tables using the normal Siebel EIM process Loading from SDH tables to Siebel base tables using UCM Batch

Manager

Interface tables Are loaded using non-Siebel software Operated on by the Enterprise Integration Manager to transfer

data to SHD tables

The UCM Batch Manager is used to move data from SDH tables to Siebel base tables With data cleaning and matching to produce “best” version

data Data is then accessible from the Siebel application’s standard

business logic displays

155


Data Loads into UCM at the Business Layer

Data loads at the business logic layer employ two distinct process managers: The Server Party Package Manager The UCM Batch Manager

The Server Party Package Manager: Reads input data Identifies the data source Dispatches the incoming Insert message Queues the data on the SDH tables

The UCM Batch Manager is used to move data from SDH tables to Siebel base tables With data cleansing and matching to produce “best” version

data Data is then accessible from the Siebel application’s standard

business logic displays

156


Managing Outbound Data Flows Records that have been inserted or updated can be selectively

published to subscribers Making “best” version record available throughout the data

ecosystem Real-time or batch mode

Outbound publication service is performed by The Server Party Package Manager for Query, Update, and

Delete operations The UCM Batch Manager for Insert Operations

157


The CRMML Messaging Protocol XML based mark-up language used by UCM

Inbound server requests Insert Query Update Delete

Outbound server features Publish and Subscribe

Formatted to provide Requesting external system ID Service(s) being requested Pointers to predefined UCM integration objects

158


Some Guidelines Initial data for UCM should be cleaned outside (Before

loading into UCM)

EIM may be more effective to load data into UCM from external systems for performance reasons

Batch publish method is more effective when send data from UCM to external systems

UCM is not complete Customer data store, UCM can not store complete customer information. UCM Contact table has limited columns only; this data is primarily for referencing customer data between multiple external systems

UUID is not numeric and big in size (about 20+ characters string), so it may not be useful as general purpose contact Id

External data quality tool like Trillium or First Logic should be used for data cleansing, UCM itself does not have data cleansing tool

159


Thank You

siebel ucm

Documents