oagi copyright oasis, 2004 resources: content assembly tutorial david rr webber chair oasis

Copyright OASIS, 2004

OAGi

Resources:http://www.oasis-open.org/committees/camhttp://www.jcam.org.uk

Content Assembly TutorialContent Assembly Tutorial

David RR Webber

Chair OASIS CAM TChttp://drrw.net

[email protected]

OAGi / NIST, Gaithersburg, April 2004.

http://drrw.net/


eBusiness XML parsing - mission profile

• Agile Interoperability

• Opening up business integration rules

• Handling local context drivers

• Validating content with semantics

• How does W3C XSD schema help?

• What does OASIS CAM provide?

• How does this work for OAGi BODs?


OASIS CAM in a nutshell• Assembly of XSD fragments into whole schema

• Provide payload structure and content definition– Local pre-deployment transaction checking

– Industry Conformance and Compliance testing

– Production validation and error handling

• Business Context Rules implementation– Simple business analyst friendly syntax and predicates

• Transaction Assembly and Integration / Formats

• Alignment of Vocabularies / Legacy Semantics


$

Time / Maturity

Semantic Interoperability Barrier

Structuring Barrier

$

$Data

Exchange Costs

Data Quality Costs

CAM – Providing Business Value


Sampling of CAM interest

• Users– British Telecom– DFAS– Smartdraw.com– XML Center NY

• Technical Committee Individuals– Adobe– Amberpoint– NIST– Sun– Seeburger– US Navy– EPR-forum– CUNY XML Center

- OAGi BODs

IV&I transactions

Purchase Order

- UBL OP70

- FIXML

- DFAS, IJI

- xCBL Telcom

CAM Templates Developed


Information Maturity Capabilities

0

10

20

30

40

50

60

Transaction Sophistication / Partner Support

EDI

XML / XSD

Schematron

CAM Templates


Agenda

• Part 1 – Basic Feature Set / Uses

– XSD Schema Assembly Techniques

• Part 2 – Content Structure Definition and Validation

– XML Structure and Data Content Validation

– Deployment / Business Context Mechanism

– Using CAM template tools to create an OAGi BOD (IV&I)

• Part 3 – Advanced Features

– Understanding Elements/Attributes/Data types/Namespace

– Using Vocabularies / Legacy Semantics

– Assembly of Business Content / Transactions

• Summary


OAGi

• XSD Schema Assembly Techniques

Part 1 – Basic FeaturesPart 1 – Basic Features


OAGi

XSD Schema AssemblyXSD Schema Assembly

Connecting standard pieces of XSD structure components together into coherent XSD schemaUsing a familiar construct to illustrate and introduce CAM featuresCompare and contrast with XSD alone


Conceptual: Data Elements build Transactions

Name

Address

email

Phone #

Company

Company

Phone #

email

Address

Name

Party

Party ID

Invoice

PO #

Line Item

Cost

CONTEXT


Example #1 - Core Components

Create Long Term Contract

Forecast Component Requirements

Send Planning Document

Place Order

Ship Materials

Customer

Arrange Payment

Supplier

Domain: Matl Mgmt

Industry: Auto •Ship Notice

Regulatory: Safety

Change Lvl

Container

Serialized ID

ProdId

Part Number

Ship’d Qty

Context Providing supply chain messaging using CCTS


Example #2 - Context Affects “Person” depending on “Role”.

CORE COMPONENT

(PERSON)

Police Officer knows he is a WITNESS

FRENCHENGLISH

Police Investigator thinksHe is a SUSPECT

NPB calls him a PAROLEE

Corrections call him aINMATE / CRIMINAL

Judge calls him aDEFENDANT

L'officier de police sait qu'il est un TÉMOIN

L'investigateur de police pense qu'il est un SUSPECT

NPB l'appelle un LIBÉRÉ SUR PAROLE

Les corrections l'appellent un DÉTENU/CRIMINEL

Le juge l'appelle un DÉFENDEUR


How do you put this all together?

Action Event

InformationRuleWhatWhy

How When

Where / Who Where / Who Where / Who

Action Event


How When

Action Event


How When


Linking Person Scenario Details to Real World

English or French Text DocumentHTMLXML SchemaPDF eForms

CORE COMPONENT

BUSINESSINFORMATION

ENTITY

SYNTAX BIEEXPRESSION

USAGECONTEXT

Context Key:UID:RCMP023079:00Geo-Political Context: English or FrenchLegislative Context: Narcotics LegistationIndustry Context: NAICS 9221Process Context: HC 3515Process Role Context: Police ChiefSupporting Role Context: RCMP Investigator Product Context: Case Management SystemSystems Capabilities Context: XML Schema

1. Example: AGE

Syntax Independent

Syntax Bound

2. Becomes: PERSON.BIRTH.DATE of Suspect

3.Is expressed in an XML schema fragment


Using CAM Template – XSD assembly

• Template consists of the following parts

– Header• Description, owner, version

• Context Declaration

– Structure• XSD component assembly details

– Business Context use rules

1

2

3

Example


Comparing XSD import to as:include• XSD approach

– Uses import mechanism to combine XSD syntax

– Can name parent element blocks from import and build structure with them

– Can rename parent element(s) with new name(s)

• CAM approach– Uses as:include predicate that retrieves XSD fragments

– Can use predicates to control cardinality

– Can exclude parts of tree, elements or attributes

– Can use rules driven by context statements

– Can include CAM sub-assembly structure(s)


• XSD <import> selection factors– Works with XSD parser standalone– Good for well defined static structures– Tools available to help create syntax and visualize the result– Complex schema may not be supported by tools such as WSDL

• CAM as:include selection factors– Provides finer granularity of control on structure members– Can use context driven selection– Requires CAM processor– Output XSD results can be used at design time or runtime with

tool that will consume the resultant simple resultant XSD– CAM V1.0 does not parse and interpret the XSD fragments

Choosing which approach?


XSD Schema Assembly Summary

• Provides one discreet use for CAM templates• Useful for documenting business rules around

selection of components of XSD structure• Works best for relatively stable transactions with

limited business permutations where XSD provides enough functionality

• Preferred if the deployment target is based around an XSD parser as the prime validator / consumer of the resultant schema.

• May solve XSD handling issues by providing fully resolved simple XSD for downstream processes / WSDL / SOAP environments


OAGi• XML Structure and Data Content Validation

• Deployment model + Business Context Mechanism

• Using CAM template tools to create an OAGi BOD

Part 2 – Content StructurePart 2 – Content Structure


OAGi The most common need is to define a transaction structure - and to then test that a given piece of XML actually conforms correctly to those business rules (but I thought XSD does all that?)

XML Structure + ValidationXML Structure + Validation


How does CAM augment schema?• Can have multiple structure instances selected by context• Can include structure from sub-assembly of components• Enhanced element semantics beyond simply datatyping• Full code lists functionality beyond NMTOKEN• Ability to leverage semantics at attribute level consistently with

elements• Ability to version content model components• Link to industry dictionaries / registries• Link to web service components for external checks • Ability to assemble transaction instances• Neutral approach - can handle non-XML / XSD formats


Technology Foundation

• Simple XML V1.0• XPath• Neutral approach to

structure content• Assertion predicates

excludeAttribute()excludeElement()excludeTree()makeOptional(), makeMandatory()makeRepeatable(), setLimit()setRequired()setChoice()setId()setLength()setMask()setValue()restrictValues()restrictValuesByUID()useAttribute()useChoice()useElement()useTree()useAttributeByID()useChoiceByID()useElementByID()useTreeByID()lookup()startBlock(), endBlock()


Example #3 – Sporting Goods Ordering

• This example illustrates the use of rule predicates both inline and in the business context section.

• The reference XML structure layout is directly inside the CAM template itself– (any associated XSD structure is used indirectly

as a design reference only)• Different XML structure components are

selected based on contextView Example


OAGiAn XSD schema structure definition provides a variety of permutations for its target XML structure. CAM provides the means to select the XML combination that matches the actual business context.

Business Context MechanismBusiness Context Mechanism


• Context is the pervasive driver to effective engineering

• Drives dynamic process configuring and control

• Ensures accurate information capture, packaging and delivery

• Key to correct relationships between partners in a collaboration

Lack of context control mechanisms is the most prominent reason why legacy e-Business systems are difficult and complex to extend and support

Date: circa 15681 : the parts of a discourse that surround a word or passage and can throw light on its meaning2 : the interrelated conditions in which something exists or occurs

Date: circa 15681 : the parts of a discourse that surround a word or passage and can throw light on its meaning2 : the interrelated conditions in which something exists or occurs

Understanding about Context


Results forgiven context andaction

Context Statements

How Context is applied

CAM Template

+

CAM gets passed context frombusiness use

Generates actual XML

instancedefinition to be used

XML transaction

contentprocess

and compare

1

23

4

5


External Business Context Mechanism• Link between messaging layer, business

process layer and assembly layer.

<ebContext UIDref='SDIR03400' interchangeID='123456789' BPMref='ABC123456:01‘ CPAref='ABC012345‘ xmlns:as="http://www.oasis-open.org/committees/cam"> <header> <description>An example context instance</description> <version>1.0</version> <language refcode='eng' codelist='ISO639-2' name='English'/> <usage>CAM</usage> <usage>BPM</usage> </header>

<conditions> <condition name="Country" value="USA" as:member="USA,CA,MX"/> <condition name="Catalogue" value="A2003-Q1" as:setLength="8" as:setMask="XNNNN-QN" as:UIDreference="SGIR:030451"/> <condition name="itemType" value="nonperishable" label="Item type:“ as:member="nonperishable,perishable,refridgerated,fragile"/> </conditions></ebContext>

Context Variables Definition


ContentAssemblyTemplate

Configure the Business Exchange Rules

XMLinstance

structure(s)

Business context rules

+Content Reference rules

+Data Validation rules

+External Mapping rules

Logical business transaction can have more than one structure layout

can add inline rules directly into structure

makeOptional()excludeTree()

item type="noun" name="Agency/@AgencyID" UIDReference="ukt000910"

<conditional expression="'//SupplierID' and lookup(value,'SGIRWSDL:supplierID_check')"<MapRule output="type" input="Sales/Company/Year/Qtr/Product@type"/>

Simple Declarative Statements

BPMStep

transactiontemplate


Moving to deployment

Business agreement

Drives documentexchanges


Context Summary

• Context variables defined in CAM template header as needed

• Optional external XML context instance document can pass values to CAM processor from a business process step

• The external XML context instance can be shared with other process components that may require knowledge of context – e.g. messaging services, business process engine.

• Registry can provide storage for the artifacts involved – business process script, CAM template, XML context instance, CPA instance – in a catalogue entry.


OAGiWorked example – creating an OAGIS BOD template

Using CAM toolsUsing CAM tools


Summary of steps required

• Pick a target XML transaction instance– Tutorial example is IV&I SyncQuantityOnHand BOD

• Insert sample XML structure instance into the CAM template

• Build structure control predicate statements for use in Business Context control section

• Build content reference statements for each unique element within the XML transaction

• Optionally add data validation rules• Complete template by testing against sample


Example #4 - IV&I SyncQuantityOnHand

• Template consists of the following parts– Header

• Description, owner, version

• Context Declaration

– Structure• XML instance of BOD

– Business Context use rules

– Content Reference statements

– Data Validation rules

Example

1

2

3

4

5


Using the CAM Wizard Tools

• Written in JavaScript• Uses IE5.5+ with MS-XML4.0 parser• Exploits the built-in XML DOM handler• Easily extensible• Writes examples to C: drive \temp directory

DemonstrationSelect Sample XML (for use in Wizard)<SyncQuantityOnHand xmlns="http://www.aiag.org/oagis/ivi" xmlns:oa="http://www.openapplications.org/oagis" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.aiag.org/oagis/ivi ../BODs/SyncQuantityOnHand.xsd" revision="8.0" environment="Production" lang="en-US"> <ApplicationArea>

<oa:CreationDateTime>2003-11-10T11:30:47-08:00</oa:CreationDateTime><oa:BODId>200311101130QOH442</oa:BODId><Sender>

<oa:Component>SV Poller</oa:Component><oa:Task>4.1012</oa:Task><oa:Confirmation>0</oa:Confirmation><LogicalId qualifier="DUNs">033987830</LogicalId>

</Sender><Receiver>

<LogicalId qualifier="DUNs">006927180</LogicalId></Receiver>

</ApplicationArea> </SyncQuantityOnHand>

View Results


Running the jCAM processor

• Available from : http://www.jCAM.org.uk

Test jCAM


More CAM toolsVisualScript Schema Designer

• Allows you to design an XML structure

• Automatically generates the XSD for the structure

• Automatically outputs the matching CAM template

• Creates two styles of documentation

• HTML

• XML

– http://www.visualscript.com

• JAXE open source XML editor– http://jaxe.sourceforge.net

– is configurable via the CAM.xsd


OAGi• Using elements / attributes / datatyping / namespaces • Alignment of Vocabularies• Multi-lingual support• Assembly of Transaction Content

Part 3 – Advanced FeaturesPart 3 – Advanced Features


OAGiUsing Elements / Attributes / Data types / Namespace techniques and managing them

Understanding XML partsUnderstanding XML parts


Elements / Attributes / Data types

• While these are at the heart of XML syntax, they are also in many ways an odd-ball trio.

• If you ask ten people about these, you will get ten different views, or more!

• How do you get to consistency of use and sensible use standards?

• How do you apply versioning and reuse?

• CAM allows you to treat structure elements and attributes consistently and apply robust data typing and validations to them equally as nouns.


Aligning Terminology and Usage

• XML allows for freeform naming of content tags• CAM provides the means to link those tag names

to standard industry terms and definitions within a formal vocabulary

• This also means that multi-lingual support is built-in

• You can also version those references• Registry SCM team is defining noun semantics


Capturing Noun Semantics• Draft layout for

storing noun details in registry

See: OASIS Registry Semantic Content Management sub-committee work (SCM SC)

Can use DesignerTools to automatically generate from XSD model


Reference Nouns from CAM template

• Using the UID reference, and Registry alias mechanisms – associate content in the schema layout with noun semantic reference

Define Registry Alias

Associate structure items

<ContentReference> <Addressing> <registry name="OAGBOD" access="registry.oagi.net:1025" method="http-binding" description="OAGi BOD Element Dictionary Registry access"/> </Addressing>

<item type="noun" name="//DataArea/oa:Sync@confirm" UIDReference="IVI070020" taxonomy="UID" registry=“OAGBODI" setValues="'Never','Always'" /> <item type="noun" name="//EffectivePeriod@inclusive" UIDReference="IVI070025" taxonomy="UID" registry=“OAGBOD" setValues="'True','False'" />

</ContentReference> Noun semantics referencing


Noun Semantics Summary

• CAM uses a simple unique identifier system (UID) tailored to the target domain – such as ISO11179, UDEF, RosettaNet, FIX, legacy EDI and more.

• With a neutral logical labelling system you decouple away from dependency on explicit human readable names – and can resolve possible collisions.

• Facilitates a system of nouns and constructing a vocabulary that provides consistent semantic use.

• Enables libraries of structure parts and foundation for automated crosswalks between transaction documents and application stores.


Names and Namespaces• Is this a solution looking for a problem?

• How much irresolvable collision of names really happens in a given XML instance?

• Local and global declarations– Can you just include a piece of XML into another XML

structure and it always work?

• Can you work smart and build XML that is self-defining and always locally unambiguous?

• What we really need is a way of establishing logical identifiers that link consistent semantics – regardless of the tag name used!


Namespaces – when to use them?• CAM itself uses namespaces.

• The most coherent reason to use them is not to apply prefixes to every piece of content – that is wasteful, confusing and prevents easy content reuse!

• CAM uses namespaces to denote a stream of markup within markup. In other words – markup that is there only to direct how the main markup is to be used - e.g. inline predicates and assertion rules.

• This use of namespaces is simple and clear.

• A namespace prefix may also denote ownership organization location in a purely informational role.


Elements / Attributes - Summary• CAM facilitates the use of simple XML V1.0 based

structure definitions that provide very stable interoperability and compatibility

• The CAM template provides the means to express the semantics around the content in the target XML structure

• The cardinality and logic statements in CAM provide the means to assemble together a clear structure instance schema

• Removes the need to use namespaces to avoid naming collisions.

• The next section looks at the techniques and examples


OAGi

Tools that support the development of industry vocabularies for noun semantics and legacy transaction integration.

Aligning Industry VocabulariesAligning Industry Vocabularies


Noun Semantics – common foundation• Need consistent way of expressing noun

definitions across industry domains• Registry SCM provides path to that• Existing industry domain vocabularies can be

easily populated into Registry using the noun semantics layout

• Once loaded Registry provides powerful set of tools to aid alignment – associations, classification, ontologies, discovery

• Alignment is two steps – internal, then external


Registry Roles “A semantic registry is the key foundation block upon which global electronic business exchanges will be built”.

BusinessAnalyst

Web Interface

InformationProfiles

Distribution

Administration

Program LevelSemantics

Information Engine

Industry Business StandardsRegistry Protocol StandardsebXML, ISO11179, OASIS,

OAGi, RosettaNet, UBL, UDEF

XML

Software Developers

Interchange

CAM


OAGiIn addition to the validation of content CAM provides the means to create transaction and information formats as an output.

Assembly of ContentAssembly of Content


Multiple Structure Layouts

• The AssemblyStructure section of the CAM template permits the definition of more than one structure

• Each structure can be labelled• The ExternalMapping section of the CAM

template can be directed to use a specific structure, or map between two structures

• There is an optional merge substitution feature that allows CAM to target any structured content layout


Mapping to XML targets in CAM template

<ExternalMapping> <ContentAssociation> <Description>OAG BOD target</Description>

<InputSource type=”XML” location=”#sourceXMLdata” /> <OutputStore type=”XML” location=”#targetBODlayout”/> <RulesSet> <MapRule output=“/OrderItem" input="@STARTGRP()"/> <MapRule output=“/OrderType" input="Sales/Company/Year/Qtr/Product@type"/> <MapRule output=“/OrderItem/Description" input="@trim(Sales/Company/Year/Qtr/Product/Item@name)"/> <MapRule output=“/OrderItem/ProductManufacturer" input="Sales/Company/Year/Qtr/Product/Item@manufacturer"/> <MapRule output=“/OrderItem/ShipmentValue" input="Sales/Company/Year/Qtr/Product/Item@value"/> <MapRule output=“/ReceivedQuantity" input="Sales/Company/Year/Qtr/Product/Item@sold"/> <MapRule output=“/OrderItem" input="@ENDGRP()"/> </RulesSet> </ContentAssociation></ExternalMapping>

Interface to / from XMLstructures in template


External Mapping to SQL targets

<ExternalMapping> <ContentAssociation> <Description>Product List</Description> <InputSource/> <OutputStore type=”SQL” location=”product_table”/> <RulesSet> <MapRule output="Products_List" input="@STARTGRP()"/> <MapRule output="type" input="Sales/Company/Year/Qtr/Product@type"/> <MapRule output="name" input="@trim(Sales/Company/Year/Qtr/Product/Item@name)"/> <MapRule output="manufacturer" input="Sales/Company/Year/Qtr/Product/Item@manufacturer"/> <MapRule output="value" input="Sales/Company/Year/Qtr/Product/Item@value"/> <MapRule output="sold" input="Sales/Company/Year/Qtr/Product/Item@sold"/> <MapRule output="Products_List" input="@ENDGRP()"/> </RulesSet> </ContentAssociation></ExternalMapping>

Interface to / from SQL


Example Merge Template to HTML<Structure ID==”#htmlReport”><![CDATA[<HTML><HEAD><META NAME="Author" Content="OASIS CAM"> <TITLE>Monthly Orders Report</TITLE></HEAD><BODY><H1><FONT COLOR="blue">Monthly Orders Report</FONT></H1><P>#as:REPEAT#;<H2><FONT FACE="Arial ">Month: #as:Month#;</FONT></H2><P><TABLE>#as:REPEAT#; <TR> <TD WIDTH="20%"><FONT FACE="Arial">#as:name#;</FONT></TD> <TD WIDTH="20%"><FONT FACE="Arial">#as:type#;</FONT></TD> <TD WIDTH="20%"><FONT FACE="Arial">#as:manufacturer#;</FONT></TD> <TD WIDTH="20%"><FONT FACE="Arial">#as:sold#;</FONT></TD> <TD WIDTH="20%"><FONT FACE="Arial">#as:value#;</FONT></TD> </TR>#as:ENDREPEAT#;#as:IF (Sales/Company/Year/Qtr/Month < "12" )#; </TABLE>#as:ENDIF#;#as:ENDREPEAT#;</TABLE></BODY></HTML>]]></Structure>

CAM structure reference

Inline merge commands

Target HTML syntax

Field substitution references


OAGiResources and Planning

SummarySummary


Summary: The Holy Grail of e-Business• Automatic information integration has been the

Holy Grail of e-Business systems since before XML was conceived.

• Lesson learned - three needs are paramount: – the ability to design transactions consistently, – the ability to document their usage in a clear way and then – the ability to drive software that can apply rules and test

information content to ensure correct compliance. • CAM approach provides three critical abilities:

– documentation of business interchange transactions, – design-time assembly support with verification, – and runtime checking of information content.


Available Components / Resources

• jCAM component– http://jcam.org.uk

• VisualScript editor for CAM templates– http://www.visualscript.com

• JAXE editor with CAM XSD– http://www.jaxe.org

• OASIS CAM TC– http://www.oasis-open.org


Next Steps

• Work in progress: – Formal Publication of OASIS Specification– Continue development of jCAM processor– Library of CAM templates by industry– Integration with OASIS Registry assembly– Building of noun semantic dictionaries


Get involved in OASIS CAM

• Download the specs from www.oasis-open.org• Open source resources www.jcam.org.uk • Encourage your industry group to adopt CAM

templates for their standards• Specify CAM-compliance from providers


OAGi

OASIS – Content Assembly Mechanism

April 2004

http://www.oasis-open.org/committees/cam

Thank YouThank You

www.oasis-open.orgxml.coverpages.orgwww.xml.org

oagi copyright oasis, 2004 resources: content assembly tutorial david rr webber chair oasis

Documents

process context

frenchlegislative context

process role context

xsd syntaxcan

xsd aloneconceptual

geopolitical context

w3c xsd schema help

oasis cam