1

Semantics empowered Physical-Cyber-Social Systems for

EarthCubePresentation at the EarthCube Face Face-to-Face Workshop of Semantics & Ontologies

Workgroup:  April 30-May 1, 2012, Ballston, VA.

Amit Sheth

Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled ComputingWright State University, Dayton, OH, USA

http://knoesis.org

Special thanks & contributions: Cory Henson, Pramod Anantharam

Web (and associated computing) is evolving

Web of pages - text, manually created links - extensive navigation

2007

1997

Web of databases - dynamically generated pages - web query interfaces

Web of resources - data, service, data, mashups - 4 billion mobile computing

Web of people, Sensor Web - social networks, user-created casual content - 40 billion sensors

Web as an oracle / assistant / partner - “ask the Web”: using semantics to leverage text + data + services - Powerset, Siri, Watson

Sem

antic

Tec

hnol

ogy

Use

d

Computing for Human Experience

Keywords

Patterns

Objects

Situations,Events

Enhanced Experience,Tech assimilated in life

Web 1.0

Web 2.0

Web 3.0

3

Sensors everywhere ..sensing, computing, transmitting

• 2009: 1.1 billion PCs, 4 billion mobile devices, 40+ billion mobile sensors (Nokia: Sensing the World with Mobile Devices)

• 6 billion intelligent sensors– informed observers, rich local knowledge

Christmas Bird Count

Data & Knowledge Ecosystem

4

Data Mining

Knowledge Discovery

Understanding & Perception

IntegrationSearch

Analysis (eg Patterns)

Browsing

Insight

Situational Awareness

Decision Support

Transactional DataObservational Data

Multimedia Data

Experimental Data

Textual Data: Scientific Literature, Web Pages, News, Blogs, Reports, Wiki, Forums, Comments, Tweets

Structured,SemistructuredUnstructuredData

OGC SWE

SSW/W3C-SSN

Semantics as core enabler, enhancer @ Kno.e.sis

5

15 faculty~50 PhD students

Excellent Industry collaborations (MSFT, GOOG, IBM, Yahoo!, HP)

Well fundedExceptional Graduates

Multidisciplinary:Health/ClinicalBiomedical Sc

Social Sc…

TextMultimedia Content

and Web Data

Metadata Extraction

Patterns / Inference / Reasoning

Semantic Models

Meta data / Semantic Annotations

Relationship Web

SearchIntegrationAnalysisDiscoveryQuestion AnsweringSituational Awareness

Sensor Data

RDB

Structured and Semi-structured Data

Knowledge Enabled Information and Services Science

From simple ontologies

Knowledge Enabled Information and Services Science

Drug Ontology Hierarchy (showing is-a relationships)

owl:thing

prescription_drug

_ brand_na

me

brandname_unde

clared

brandname_comp

osite

prescription_drug

monograph_ix_cla

ss

cpnum_ group

prescription_drug

_ property

indication_

property

formulary_

property

non_drug_

reactant

interaction_proper

ty

property

formulary

brandname_indivi

dual

interaction_with_prescriptio

n_drug

interaction

indication

generic_ individua

l

prescription_drug_ generic

generic_ composit

e

interaction_ with_non_ drug_react

ant

interaction_with_monograph_ix_class

Knowledge Enabled Information and Services Science

to complex ontologies

Knowledge Enabled Information and Services Science

N-Glycosylation metabolic pathway

GNT-Iattaches GlcNAc at position 2

UDP-N-acetyl-D-glucosamine + alpha-D-Mannosyl-1,3-(R1)-beta-D-mannosyl-R2 <=>

UDP + N-Acetyl-$beta-D-glucosaminyl-1,2-alpha-D-mannosyl-1,3-(R1)-beta-D-mannosyl-$R2

GNT-Vattaches GlcNAc at position 6

UDP-N-acetyl-D-glucosamine + G00020 <=> UDP + G00021

N-acetyl-glucosaminyl_transferase_VN-glycan_beta_GlcNAc_9N-glycan_alpha_man_4

Knowledge Enabled Information and Services Science

A little bit about semantic metadata extractions and annotations

Knowledge Enabled Information and Services Science

WWW, EnterpriseRepositories

METADATA

EXTRACTORS

Digital Maps

NexisUPIAP

Feeds/Documents

Digital Audios

Data Stores

Digital Videos

Digital Images. . .

. . . . . .

Create/extract as much (semantics)metadata automatically as possible;

Use ontlogies to improve and enhanceextraction

Extraction for Metadata Creation

Knowledge Enabled Information and Services Science

Automatic Semantic Metadata Extraction/Annotation of Textual Data

Semantic Sensor Web Infrastructure

15

Semantically Annotated O&M

<om:Observation>

<om:samplingTime><gml:TimeInstant>...</gml:TimeInstant>

</om:samplingTime>

<om:procedure xlink:role="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#Sensor“

xlink:href="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#sensor_xyz"/>

<om:observedProperty xlink:href="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#temperature"/>

<featureOfInterest xlink:href="http://sws.geonames.org/5758442/"/>

<om:result uom="http//www.w3.org/2009/Incubator/ssn/ontologies/SensorOntolgy.owl#fahrenheit">42.0</om:result>

</om:Observation>

Semantic Sensor ML – Adding Ontological Metadata

16

Person

Company

Coordinates

Coordinate System

Time Units

Timezone

SpatialOntology

DomainOntology

TemporalOntology

Mike Botts, "SensorML and Sensor Web Enablement," Earth System Science Center, UAB Huntsville

Workflow Architecture for Managing Streaming Sensor Data

18

Weather Application

Detection of events, such as blizzards, from weather station observations on LinkedSensorData

Weather Application

Demos: Real-Time Feature Streams

19

Weather ApplicationSECURE: Semantics Empowered Rescue Environment

Rescue robots detect different types of fires, which may require different methods/tools to extinguish, and relays this knowledge to first responders.

Demo: SECURE: Semantics Empowered Rescue Environment

A Challenging Example Query

What schools in Ohio should now be closed due to inclement weather?Need domain ontologies and rules to describe type of inclement weather and severity.

Integration of technologies needed to answer query1. Spatial Aggregation2. Semantic Sensor Web3. Machine Perception4. Linked Sensor Data5. Analysis of Streaming Real-Time Data

20

More details in: Spatial Semantics for Better Interoperability and Analysis: Challenges and Experiences in Building Semantically Rich Applications in Web 3.0

Technology 1Spatial Aggregation

21

• What schools are in Ohio?• What weather sensors are near each of the school?

Technology 2

Semantic Sensor Web (SSW)

• What is inclement weather?• What sensors in Ohio are capable of detecting inclement weather?• What sensors are near schools in Ohio?• What observations are these sensors generating NOW?

22

Technology 3

Active Machine Perception

• Are these observations providing evidence for inclement weather?

23

Technology 4

Linked Sensor Data

• What schools are in Ohio?• What inclement weather necessitates school closings?• What sensors in Ohio are capable of detecting inclement weather?• What sensors are near schools in Ohio?• What observations are these sensors generating NOW?

24

Technology 5

Analysis of Streaming Real-Time Data

• What observations are these sensors generating NOW?

25

Demos

• Real-Time Feature Streams• SECURE (presentation:• SECURE: Semantics Empowered resCUe EnviRonmEnt )Amit• Trusted Perception Cycle• Sensor Discovery on Linked Data• Semantic Sensor Observation Service (SemSOS)

Related Talk• Spatial Semantics for Better Interoperability and Analysis: Challeng

es and Experiences in Building Semantically Rich Applications in Web 3.0: Amit Sheth delivers talk at the 3rd Annual Spatial Ontology Community of Practice Workshop: Development, Implementation and Use of Geo-Spatial Ontologies and Semantics, 3 October 2010, USGS, Reston, VA.