semantic types webinar slides

8/7/2019 Semantic Types Webinar Slides

http://slidepdf.com/reader/full/semantic-types-webinar-slides 1/21

expressor Thought leadership webinarSemantic Types: Making ETL data mapping simpler and easier to maintain

• Bill Kehoe, Chief Architect

• Wiqar Chaudry, Product Manager

www.expressor-software.com



copyright 2011 expressor software corporation2

Bill Kehoe

Bill Kehoe is a founding engineer at expressor and has been a keydeveloper since the original 1.0 version of the product. He is now achief architect providing technical leadership on all aspects of productengineering. Previously Bill was an architect at Blue Agave Software where he wasthe lead developer for the data sub-system of a supply chainmanagement product. Bill also held a senior architect role at Versataand a senior developer and program manager at Sybase. At Sybase,he architected and led the development for SQL Debug, a clientserver application for debugging SYBASE Transact-SQL storedprocedures.

Bill graduated Magna Cum Laude from Tufts University in CivilEngineering and is a member of the Tau Beta Pi Engineering honorsociety. He has also done post graduate work at Harvard University.




Today’s Agenda

expressor Studio 3.1 product overview Semantic Types

What are they?

Sample Application

Studio DemoMultiple data source formats mapped to a

common semantic type

2011 product roadmap Q&A




expressor Studio 3.1

Free & downloadable

Integrated visual development studio for ETLapplications

Built-in productivity tools and wizards– Automatically capture DB connectivity details

– Automatically capture metadata

– Access to standard business rules, e.g. date formatconversions

– Automatic type conversion

Just-in-time error notification and help

Graphical library system for reusing design

assets Fast data processing engine

Semantic Types– Field name and type standardization capabilities

– More to come in upcoming 3.x releases

www.expressorStudio.com




expressor

Semantic Types




Traditional Point-to-point vs. Canonical Mapping

Point-to-point Mapping(traditional ETL)

Semantic Types improvetime-to-value through

greater reuse andsimplified data mappings

Canonical Mapping(expressor)

SemanticType


http://slidepdf.com/reader/full/semantic-types-webinar-slides 7/21copyright 2011 expressor software corporation7

What is a Semantic Type?

An abstract data interface independent of:

Physical formatting details such as delimiters,character encoding, field formats like date, time,currency, etc.

Defines the logical structure of data, free ofpositional constraints (i.e. field order independent)

Enables rule expression (constraints, data quality,transformations and derivations) independent of

external interface “baggage” Enables rapid application assembly

Eases Data Governance / Data Lineage Tracking



Physical vs. Semantic

Physical metadata Semantic metadata

FieldColumn

[Leaf] Element

[Attribute of]

Atomic Semantic Type

CSV RecordDatabase Table

Array

Composite Semantic Type

[Non-Leaf] Element

Nested Table

Nested Composite Type(i.e. Composite Attribute)



Recurring Data Integration challenge

One logical target schema

Apply common set of rules to ensure logical dataintegrity, but …

Input data is messy!

Dozens of external formats to consume

Data quality varies depending on data source

Goal: Easily assemble a single, maintainableapplication that can consume all sources of dataand supports extensions over time



Sample application: Exploration expenses

Data warehouse for analyzing energyexploration expenses

Sub-contractors used for site work

One target data warehouse table

One set of validation rules

Multiple, contractor-specific expense dataformats

Contract_ID : integer

Item_ID: varchar(20)Item_Description : varchar(1024)

ExpType : varchar(10)

Amount : decimal(10,2)

StartDepth : Integer

EndDepth : integer

ExpenseDate : date

EXPENSE

Contractor1

Contractor2

Contractor3

Validation

Rules



ContractExpense

Semantic Type example: ContractExpense

ContractIdentifierString

ItemIdentifierString

ItemDescriptionString

ExpenseTypeString

MonetaryAmountDecimal

BookDateDatetime

DepthInteger

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth




Contractor 1 Data Format

ContractId ItemId ItemDescription ExpType Amount Date StartDepth EndDepth

Schema: Contractor1

ContractExpense

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType

MonetaryAmount

BookDate

Depth

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth



ContractExpense

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType

MonetaryAmount

BookDate

Depth

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth

Schema: Contractor1







ContractExpense

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType

MonetaryAmount

BookDate

Depth

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth





Contract Date ItemId Amount ItemDescription ExpType StartDepth EndDepth

Schema: Contractor1

Schema: Contractor2




ContractExpense

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType

MonetaryAmount

BookDate

Depth

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth






Schema: Contractor1

Schema: Contractor2

Different fieldorders, date andnumeric valueformats




ContractExpense

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType

MonetaryAmount

BookDate

Depth

ContractIdentifier

ItemIdentifier

ItemDescription

ExpenseType Amount

ExpenseBookDate

EndDepth

StartDepth

Schema: Contractor2






9.999,99




expressor Studio – Multiple data source formats

mapped to a common Semantic Type

Demo

On demand Webinar Link: http://bit.ly/fW27F7

http://bit.ly/fW27F7






Download our

Semantic TypesWhite paper

www.expressor-

software.com/semantic-types

Semantic Types white paper

http://www.expressor-software.com/semantic-types










Thank You!

Questions?



[email protected]

Thank You!

www.expressor-software.com

semantic types webinar slides

Documents