don hurst, zymogenetics sarmad pirzada, hybrid data systems

Use of selected Third Party XML Tools in Use of selected Third Party XML Tools in Conjunction with the Importing, Exporting and Conjunction with the Importing, Exporting and

Manipulation of XML Files within the SASManipulation of XML Files within the SAS®®SystemSystem

------------------------------------------------------------------------------------------------------------WUSS, 2007WUSS, 2007

Don Hurst, Don Hurst, ZymogeneticsZymogeneticsSarmad Pirzada, Hybrid Data SystemsSarmad Pirzada, Hybrid Data Systems

22

DisclaimerDisclaimer

The material contained in this The material contained in this presentation represents the opinions of presentation represents the opinions of the presenters and should not be the presenters and should not be construed as reflecting ZymoGeneticsconstrued as reflecting ZymoGenetics’’position or opinion. position or opinion.

33

XML into SASXML into SAS

1.1. Getting an XML files into SASGetting an XML files into SAS2.2. SAS and nonSAS and non--SAS tools for SAS tools for

manipulating XML filesmanipulating XML filesa.a. XML XML MapperMapperb.b. AltovaAltova XML SpyXML Spyc.c. oXygenoXygen XML EditorXML Editord.d. Exchanger XML Exchanger XML LiteLite V3.2 editorV3.2 editore.e. W3C Validation checkerW3C Validation checker

44

XML TermsXML Terms

–– XML: Extensible Markup LanguageXML: Extensible Markup LanguageNew method of for web applications and data New method of for web applications and data transfertransferwww.w3c.orgwww.w3c.org standard for creating XML tags for standard for creating XML tags for transferring and interpreting business filestransferring and interpreting business files

–– TAG: <TAG: <subjnumsubjnum>2008</>2008</subjnumsubjnum>>–– DTD: Data type definition DTD: Data type definition

collection of tags collection of tags parent child relationship parent child relationship

55

SGML SGML –– XML RelationXML Relation

66

What Does XML Look Like?What Does XML Look Like?

<?xml version="1.0" encoding="ISO<?xml version="1.0" encoding="ISO--88598859--1" ?> 1" ?> <!>

<note><note><to><to> oveove </to></to><from><from> JaniJani </from></from><heading><heading> ReminderReminder </heading></heading><body><body> Don't forget me this weekend!Don't forget me this weekend! </body> </body> </note></note>

77

Another XML ExampleAnother XML Example

<?xml version="1.0" encoding="ISO<?xml version="1.0" encoding="ISO--88598859--1" ?>1" ?><<breakfast_menubreakfast_menu>>

<<foodfood>><name><name>Belgian WafflesBelgian Waffles</name> </name> <price><price>$5.95$5.95</price> </price> <description><description>two of our famous Belgian Waffles with two of our famous Belgian Waffles with

plenty of real maple syrupplenty of real maple syrup</description> </description> <calories><calories>650650</calories> </calories>

<</food/food >></</breakfast_menubreakfast_menu>>

88

Every XML File Should HaveEvery XML File Should Have

There is a There is a ““beginningbeginning”” and and ““endend””container tagcontainer tagThere is a hierarchy of items within There is a hierarchy of items within the the ““foodfood”” segmentssegmentsEvery segment and field begins and Every segment and field begins and ends with a tagends with a tagIt should be wellIt should be well--formedformed

99

XMLXML ErrorError(Internet Explorer)(Internet Explorer)

The XML page cannot be displayed The XML page cannot be displayed Cannot view XML input using XSL style sheet. Cannot view XML input using XSL style sheet.

Please correct the error and then click the Please correct the error and then click the RefreshRefresh button, or try again later. button, or try again later.

End tag 'End tag 'FfromFfrom' does not match the start tag ' does not match the start tag 'from'. Error processing resource 'from'. Error processing resource 'http://'http://www.somwhereovertherainbow.com/xwww.somwhereovertherainbow.com/xml/note_error.xmlml/note_error.xml......

<from> <from> JaniJani </</FFfromfrom> > --------------------------^̂

1010

Correcting the XML ErrorCorrecting the XML Error

Several methods:Several methods:––Note pad and redisplaying in IENote pad and redisplaying in IE––W3C consortium toolsW3C consortium tools––3d party XML editing tools3d party XML editing tools

1111

Representing a Hierarchy in SASRepresenting a Hierarchy in SAS

Visual perspective of the hierarchical Visual perspective of the hierarchical XML file structure small part of XML XML file structure small part of XML SAS datasets are rows/columns, e.g. SAS datasets are rows/columns, e.g. observations/variablesobservations/variablesDefaults that exist when an XML file is Defaults that exist when an XML file is read into SASread into SAS3 levels can be handled without 3 levels can be handled without extensive mapping extensive mapping

1212

<?xml version="1.0" encoding="windows-1252" ?><TABLE>

<DEMOG><SUBJID> 200 </SUBJID><AGE> 58 </AGE><SEX> Female </SEX><RACE> Asian </RACE><BIRTHDA> 1948-04-01 </BIRTHDA>

</DEMOG></TABLE>

An Example

1313

How do I get that XML file into How do I get that XML file into SAS?SAS?

libname in1 xml‘Y:\subject.xml' ;proc copy indd=in1 outdd=work ; run;

1414

What does it look like in SAS (Proc Print)?

Demographics “DEMOG” XML file represented in SAS

Obs BIRTHDA RACE SEX AGE SUBJID1 1948-04-01 Asian Female 58 2002 1965-12-18 Black Female 37 3003 1944-07-22 Other Male 59 400

Why is the “Y” left off of the “BIRTHDAY” variable name?

1515

Role of Schema/MappingRole of Schema/Mapping

XML schema tells machines how to carry out XML schema tells machines how to carry out rules defined in the schema, define the data rules defined in the schema, define the data structure, content, shared vocabularies and structure, content, shared vocabularies and semantics of XML documents.semantics of XML documents.Business rules often dictate complex hierarchical Business rules often dictate complex hierarchical relationships that are beyond the default relationships that are beyond the default capabilities of the SAS/XML enginecapabilities of the SAS/XML engineSASSAS®® XML XML MapperMapper was created for such a was created for such a purpose on mapping the XML document into purpose on mapping the XML document into SASSAS

1616

XML XML MapperMapper OverviewOverview

XML XML MapperMapper is a tool provided by the SAS is a tool provided by the SAS Institute for telling SAS how to read and Institute for telling SAS how to read and represent an XML file into SASrepresent an XML file into SASXML XML MapperMapper has been widely written has been widely written about and will be briefly covered here.about and will be briefly covered here.

1717

SAS SAS ®® XML XML MapperMapper

MapperMapper is a Graphical User Interface is a Graphical User Interface (GUI) written in Java (GUI) written in Java Allows the user to generate XML Map by Allows the user to generate XML Map by drop and dragdrop and dragAlso can show sample SAS code, table Also can show sample SAS code, table views, and contents.views, and contents.

1818

What XML What XML MapperMapper looks likelooks like

XML Window

Source Window

Map Window

1919

XML XML MapperMapper with ODM XMLwith ODM XML

XML Window

Map Window

Source Window

2020

XML XML MapperMapper with Manufacturer with Manufacturer XMLXML

XML Window

Map Window

Source Window

2121

XML XML MapperMapper ShortcomingsShortcomings

It sometimes will abort with an error It sometimes will abort with an error without any warning.without any warning.Its validation of a well formed XML Its validation of a well formed XML structure is not necessarily the same as structure is not necessarily the same as other 3other 3rdrd party tools, e.g. party tools, e.g. XMLspyXMLspy, , oXygenoXygen, , etc.etc.Map creation is very iterative and the Map creation is very iterative and the resulting map when used in a SAS resulting map when used in a SAS program can be frustrating.program can be frustrating.

2222

How to Get SASHow to Get SAS®® MapperMapper

Comes with the SAS Foundation Comes with the SAS Foundation cdcd’’ssOr, download from Or, download from support.sas.comsupport.sas.com under under base softwarebase softwareIt can be run as a standalone after It can be run as a standalone after unzipping into a folder by clicking on unzipping into a folder by clicking on ““sas.sxle.atlassas.sxle.atlas””, an executable jar., an executable jar.Meant, however, to be a plugMeant, however, to be a plug--in into the in into the SAS products of the programs.SAS products of the programs.

2323

Other NonOther Non--SasSas 33rdrd Party ToolsParty Tools

AltovaAltova’’ss XML SpyXML Spy–– Generally good for nonGenerally good for non--PharmaPharma XML filesXML files–– Heavily used in the Financial/Manufacturing Heavily used in the Financial/Manufacturing

sectorssectors–– There were some problems validating an There were some problems validating an

PharmaPharma XML file conforming to CDISC and XML file conforming to CDISC and W3C standardsW3C standards

Used MSXML engine and was not W3C compliantUsed MSXML engine and was not W3C compliantJust recently changedJust recently changed

2424

What What AltovaAltova XML Spy looks likeXML Spy looks like

2525

XML Spy Validation PortionXML Spy Validation Portion(Validation Error)(Validation Error)

Unable to show Unable to show schema:Cschema:C::\\Program Program FilesFiles\\AltovaAltova\\XMLSpy2007XMLSpy2007\\ExamplesExamples\\IndusIndustryStandardstryStandards\\PatentsPatents\\applicationapplication--example.xmlexample.xml

XML Schema has to begin with a <schema> XML Schema has to begin with a <schema> element in namespace element in namespace 'http://www.w3.org/2001/XMLSchema'!'http://www.w3.org/2001/XMLSchema'!Error location: patentError location: patent--applicationapplication--publicationpublication

2626

<<oXygenoXygen/>(r) XML Editor/>(r) XML Editor

2727

<<oXygenoXygen/>(r) XML Editor/>(r) XML Editor(Validation Error)(Validation Error)

systemid: C:\Program Files\Oxygen XML Editor 8.2\samples\svg\sales.xmlDescription: There is no schema or DTD associated with the document.

You can create an association either with the Associate Schema action or configuring in the Options the Preferences/Editor/Default Schema Association list, or by creating a Validation Scenario.

2828

XML Exchanger XML Exchanger LiteLite V3.2V3.2

2929

XML Exchanger XML Exchanger LiteLite 3.23.2(Validation of Schema)(Validation of Schema)

[[XercesXerces--J 2.7.1] Checking "J 2.7.1] Checking "converted.xsdconverted.xsd" " for Wellfor Well--formednessformedness ......

WellWell--formed Document.formed Document.

3030

www.w3c.orgwww.w3c.orgValidation CheckerValidation Checker

3131

ODM XML file to be ValidatedODM XML file to be Validated

3232

SAS System and XMLSAS System and XML(summary)(summary)

SAS System

XML Out XML In

Business Rules

The “XML In”requires a XML

document &schema for SASto properly parse

the document

3333

Common SASCommon SAS®®/XML Flow/XML Flow

Validated XML document (xml file + Validated XML document (xml file + schema)schema)Industry specific business rulesIndustry specific business rulesMapperMapper file for bringing validated XML file for bringing validated XML documents into SASdocuments into SASIndustry specific XML validation with Industry specific XML validation with schema rules in accord with W3Cschema rules in accord with W3C

3434

Concluding RemarksConcluding Remarks

Manipulate XML file data within the SAS Manipulate XML file data within the SAS system in accordance to a set of business system in accordance to a set of business rulesrules–– Depends upon the Industry group and Depends upon the Industry group and

applicationapplication–– Desired output affects the manipulation of the Desired output affects the manipulation of the

SAS representation of the XML data.SAS representation of the XML data.

3535

Concluding RemarksConcluding Remarks(continued)(continued)

SAS and representative nonSAS and representative non--SAS tools are SAS tools are available to facilitate XML file handlingavailable to facilitate XML file handling–– XML XML MapperMapper–– AltovaAltova xmlspyxmlspy editoreditor–– oXygenoXygen XML editor XML editor –– Exchanger XML Exchanger XML LiteLite V3.2 editorV3.2 editor–– W3C Validation checkerW3C Validation checker

3636

References and Further ReadingReferences and Further ReadingSASSAS®® XML XML LibnameLibname Engine 9.1.3, UserEngine 9.1.3, User’’s Guide, SAS Institute, s Guide, SAS Institute, Cary, NC, USACary, NC, USASASSAS®® Implementing CDISC data models in the SAS Metadata Implementing CDISC data models in the SAS Metadata ServerServerYehYeh, Shi, Shi--Tao. Using XML Tao. Using XML MapperMapper®® to Create an XML Map. to Create an XML Map. NESUG18 Proceedings. NESUG18 Proceedings. CisternasCisternas, M., (2004): Reading and Writing XML files from SAS, M., (2004): Reading and Writing XML files from SAS®®, , Proceedings of the 29Proceedings of the 29thth SAS Users Group International Conference, SAS Users Group International Conference, Paper 119.Paper 119.Pirzada, Sarmad. Pirzada, Sarmad. Moving XML Files within SAS® and Maintaining Segment Hierarchy. PNWSUG, 2005.ZenderZender, , CymthiaCymthia L. Creating a L. Creating a TagsetTagset Template for the SASTemplate for the SAS®® XML XML LibnameLibname Engine. TTEngine. TT--24, SAS Conference Proceedings: 24, SAS Conference Proceedings: PharmaSUGPharmaSUG 2007June 32007June 3--6, 2007, Denver, Colorado.6, 2007, Denver, Colorado.

3737

References and Further ReadingReferences and Further Reading(continued)(continued)

www.w3c.orgwww.w3c.orgsupport.sas.comsupport.sas.comwww.cdisc.orgwww.cdisc.orgwww.altova.comwww.altova.comwww.oxygenxml.comwww.oxygenxml.comwww.freexmleditor.comwww.freexmleditor.com

3838

AcknowledgementsAcknowledgements

SAS is a registered trademark of SAS SAS is a registered trademark of SAS Institute, Inc. in the USA and other Institute, Inc. in the USA and other countries. Other brand and product countries. Other brand and product names are registered trademarks or names are registered trademarks or trademarks in their respective companies.trademarks in their respective companies.®® indicates USA registration.indicates USA registration.

3939

Questions?Questions?

Don Hurst, Don Hurst, [email protected][email protected] Pirzada, Sarmad Pirzada, [email protected]@hybriddatasystems.com

don hurst, zymogenetics sarmad pirzada, hybrid data systems

Documents