the orst data model / neesgrid charles severance october 2, 2003

The ORST Data Model / NEESGrid

Charles Severance

October 2, 2003

Work to date

• Partial Conversion of ORST model– As example for NEESML

• Expect to be able to create ORST objects in Release 2.0 of NEESgrid

• Work has begun on CHEF based experiment management tool modeled after the ORST tool

Why start with the ORST Model?

• It provides good coverage of a well-understood scope

• It has consensus across multiple sites• A prototype toolset exists around the model

which is an important validation• http://nees.orst.edu/IT/data.model/• We can adopt the core elements and extend

as necessary

QuickTime™ and aTIFF (LZW) decompressorare needed to see this picture.



http://nees.orst.edu/IT/data.model/docs/v1.3a.july16.pdf









<type id="o:project" title=”Project"> <specialConditions title=”Special Conditions"/> <title title=”Title"/> <startDate title=”Starting Date"/> <o:experiment allow="o:experiment” max="unbounded" /> <o:role allow="o:role” max="unbounded" /> <o:acknowledgements allow=“o:acknowledgement” max=“unbounded” /></type><type id="o:experiment" title=”Experiment"> <status title=”Status"/> <title title=”Title"/> <shortDescription title=”Description"/> <o:facility allow="o:facility” /> </type><type id="o:experiment" title=”Experiment"> <title title=”Title"/> <shortDescription title=”Description"/> <longDescription title=”Description"/></type>

o:project

o:experiment[s]

o:role[s]

o:acknowledgement[s]

specialCondions

title

startDate

o:experiment

o:facility

status

title

shortDescription

o:facility

longDescription

title

shortDescription

<type id="o:project" title=”Project"> <specialConditions title=”Special Conditions"/> <title title=”Title"/> <startDate title=”Starting Date"/> <o:acknowledgements allow=“o:acknowledgement” max=“unbounded” /></type><type id="o:experiment" title=”Experiment"> <status title=”Status"/> <title title=”Title"/> <shortDescription title=”Description"/> <o:project allow="o:project” /> <o:facility allow="o:facility” /> </type><type id="o:facility" title=”Experiment"> <title title=”Title"/> <shortDescription title=”Description"/> <longDescription title=”Description"/></type>

o:project


specialCondions

title

startDate

o:experiment

o:project

o:facility

status

title

shortDescription

o:facility

longDescription

title

shortDescription

<type id="o:project" title=”Project"> <specialConditions title=”Special Conditions"/> <title title=”Title"/> <startDate title=”Starting Date"/> <o:acknowledgements allow=“o:acknowledgement” max=“unbounded” /></type><type id="o:experiment" title=”Experiment"> <status title=”Status"/> <title title=”Title"/> <shortDescription title=”Description"/> <o:project allow="o:project” max=“unbounded” /> <o:facility allow="o:facility” /> </type><type id="o:experiment" title=”Experiment"> <title title=”Title"/> <shortDescription title=”Description"/> <longDescription title=”Description"/></type>

o:project


specialCondions

title

startDate

o:experiment

o:project[s]

o:facility

status

title

shortDescription

o:facility

longDescription

title

shortDescription

object

When QueryingSQL:“work your way down joins”

select * from tExperiment inner join on iProjectID where iProjectID = 42

42

42

42

o:experiment

o:project

RDF-style:“Work your way out from what you want”

retrieve object of type “o:experiment” where the “o:project” = this-project

o:experiment

o:project

o:project

nsf:iniative

nsf:iniative

Translating Terminology

• One to many (project to experiments) – Reference goes in the

destination (experiment)

• Many to one (experiments to facilities)– Reference goes in the source

(experiment)

• Many to many (people to roles)– Create intermediate object to

store linkage (person-role)

object

object

object

object

object

object

object

object

object

object

object

Flexiblity using this approach

• Can extend objects when creating new objects

• Can make hierarchical structures

• Can make network structures

• Structures can coexist in the same repository

<type id=”x:project" title=”Project” extends=“o:project”> <nsf:iniative allow=“nsf:iniative” /></type><type id=“nsf:iniative” title=“Iniative”> <title title=”Title"/> <startDate title=”Starting Date"/> ….</type>

o:project


specialCondions

title

startDate

x:project


specialCondions

title

startDate

nsf:iniative

nsf:iniative

….

title

startDate

At some point we just make a new object - we can relate it to the existing object.

<type id=”x:project" title=”Project” extends=“o:project”> <nsf:iniative allow=“nsf:iniative” /> <x:project-parent allow=“x:project” /> <x:related-project allow=“x:project” max=“unbounded” /></type><type id=“nsf:iniative” title=“Iniative”> <title title=”Title"/> <startDate title=”Starting Date"/> ….</type>

o:project


specialCondions

title

startDate

x:project


specialCondions

title

startDate

nsf:iniative

x:project-parent

x:related-project[s]

nsf:iniative

….

title

startDate

x:project

object

object

At some point we just make a new object - we can relate it to the existing object.

Data Formats / Metadata Models

• There is a difference between data and meta data• Data is in a file - it has a format - some tools know

who to read and write the format– Word Autocad STEP

• Metadata is information “about” files and other things stored in our format

• Sometimes metadata is automatically extracted from files in a particular format by software which knows some or all of the format of files

object

object

RDBMS


RDBMS


Autocad

SensorML

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

object

object

object

object

object

object

object

Strategies

• Take Existing Data Models and Adopt• Build tools and data models at the same

time• Find existing tools that produce types of

data we find interesting• Build meta data extractors for important file

formats• Build converters for things like Excel

CHEF Tool ORST

• Effort to understand the ORST data model• CHEF tools are broken into services and GUI components

using a cleanly defined API• We can have different service implementations

– Test service using memory– Repository based service– RDBMs based service– Service which can integrate across both Repository and RDMBS

sources.

• The tool can be used to explore extensions to the model.• Simple service has been written GUI is being developed

QuickTime™ and aPhoto - JPEG decompressorare needed to see this picture.

Looking for Data Models…

Questions about data efforts

• Is this a data format or is this a meta data model?

• How is the data represented? Is the representation published? Can we get a copy of the documentation? Does it have a URL?

• Is the model/format extensible?

• What community developed the model/format? Who is responsible for maintaining the model/format? What kind of relationship is necessary to work with them? Are there fees?

• What applications has the model/format been used for?

• Is this model in use at NEES sites?

• What kinds of software support are there for the model/format? Are there APIs to manipulate the model?

STEP

• ISO 10303, commonly known as STEP (Standard for the Exchange of Product model data)

• Base standards plus “profiles” - (i.e. “how we use step around here”)– http://pdesinc.aticorp.org/ http://www.steptools.com/.

– AP-227 for Ship Piping Usage Guide v0.1

– http://www.steptools.com/support/stdev_docs/express/part227/index.html

• We could develop our own STEP “profile”

http://www.steptools.com/

http://www.steptools.com/support/stdev_docs/express/part227/index.html





How to prioritize model exploration and development

• Focus on the following areas:– Areas where we have or are building tools

– Areas where we already have incoming data in some format

• Christopher tells me that they have equipment that emits SensorML

– Build the model through experiment based deployment - solve real problems in an open way and see if (with some adaptation) the solutions apply more broadly (i.e. Minnesota )

Summary

• The ORST model is an excellent place to start because it is small enough to understand and well documented so we can actually build tools and achieve some interoperability

• We will quickly find places in which we need additional capabilities (i.e. the Minnesota report)

• We need to prioritize model work to work on the most useful portions first

the orst data model / neesgrid charles severance october 2, 2003

Documents