sep 22, 2003 data handling interface enhancing capabilities at the iris dmc iris data management...

Sep 22, 2003

Data Handling Interface Enhancing Capabilities at the IRIS DMC

IRIS Data Management Center

Sue Schoch

[email protected]

Sep 22, 2003 EMICES/ORFEUS Workshop

Overview

• Background of DHI• Data Handling Interface• DHI Clients


Data Sources

•DMS Virtual Private Network•Frame Relay

•ASL

•IDA

•AFTAC

•NEIC

•FTP sites•ASL

•IDA

•others

•Real Time Links•Many, many, many sites


Data Storage

OracleDatabase

BUDBuffer of

Uniform Data

ARCHIVEPONDPool of

Network Data


BUDReception of Data in Real Time

IUH2

ICLISS

ER

RE

WY

IE

AR

MB

LBUSUW

UO

XX SC NM

UU

PR

EarthwormWaveserver

PN

BBArray

II

NRTSLD

CI NN

AKAZ

KNKZ

IM

GT

AU

CD-1

EQ

NL

AntelopeORB

MN

GE

SEEDLink

BUD

BUD


BUD Stations (5/29/2003) 23 Networks 763 Stations 4057 channels 10.4 gigabytes per day 3.8 terabytes per year

BUD


ARCHIVE

ARCHIVE

BUD Removal14 days EH

62 days other

BATS7 days after

Real timeBUD ARCHIVE

QC’d DATAPASSCAL


ARCHIVEMass Storage System

StorageTek PowderhornInside a Powderhorn

7-2/3’ high x 10-2/3’ Diameter

ARCHIVE


POND event based products

BUD

ARCHIVE

POND

FARMProducts

SPYDERProducts

SPYDERBuilding

FARMBuilding

POND


Oracle Database

OracleDatabase

BUD ARCHIVE POND

Metadata

EventCatalogs

HumanInput

OperationStats

OracleDatabase


Current Data Access

SeismiQuery

NetDC

BREQ_FAST

WEED

WILBER II

BUD Tools

Dataless

SPYDER® Building

FARM Building

Limited QC Continuous DataMixed Quality Continuous Data

LEGEND

Limited QC Event Windowed DataMixed QC Event Windowed Data

Metadatain Oracle

SPYDERBUD

FARMARCHIVE

POND


Need for new access methods• The amount of seismic data is increasing at a rate much faster

than funding levels• To improve efficiencies, more automated, electronic methods

need to be utilized• Platform Independent object-oriented solutions are desirable

• Portable Code with JAVA• Software reuse

• Only 3 primary distributed computing models existed• Enterprise Beans

• JAVA only• DCOM

• Microsoft• CORBA (Common Object Request Broker Architecture)

• Object Management Group• Language and platform neutral

• FISSURES - Already defined need for objects and interfaces in 1999


Overview



Data Handling Interface

New Concept in Distributing Data

• Electronic data handling• Moving from transfer by physical media

• Data flow in real time• Up-to-data metadata• Uses object-oriented techniques

• JAVA• CORBA


FISSURESFramework for the Integration of Seismological Software for University Research in Earth Sciences

• FISSURES model• Defines seismic classes

• Methods• Attributes

• Defines the relationships between objects

Seismic Object


IDL Interface Definition Language

• IDL • Describes objects, their attributes and methods• IDL compiler

• converts IDL to specific programming language• creates skeletons and stubs

• True multi-tier• Client objects write to a stub• Server objects write to a skeleton

Seismic Object

IDL


CORBACommon Object Request Broker

• The mechanism that connects objects over a distributed network

• Implements many services

• Object discovery• Event services• Security• Authentication• etc.

Seismic Object

IDL

Seismic Object

Seismic Object

IDL

IDL

CORBA


IDLNetworkService

ORB

IDLWaveformService

ORB

DHI Services at the IRIS DMC

IDLEventService

ORB


DHI Data Center Services

• Network information• Networks, Stations, channels, instrumentation, data inventory• Response representations

• Event information• Location, magnitudes, etc.• Moment tensors

• Waveforms• Continuous Data from Archive• Event data products

• FARM• SPYDER®

• BUD


IDLEventService

ORB

DHI Services and Client Programs

IDLWaveformService

ORB

IDLNetworkService

ORB

IDL WaveformService

ORB

IDL NetworkService

ORB

IDL EventService

ORB

Client1

Client2


DHI Services Interface to Data

SeismiQuery

NetDC

BREQ_FAST

WEED

WILBER II

BUD Tools

Dataless

SPYDER® Building

FARM Building

OracleDatabase

SPYDERBUD

FARM

ARCHIVE

EventService

NetworkService

WaveformService

POND


IfEvent Service

• Interfaces• EventFinder• EventAccess• EventChannelFinder


IfEvent: EventFinder Interface

• query_events• get_by_name


IfEvent: EventAccess Interface

• get_attributes• get_origins• get_origin• get_preferred_origin


IfEvent: EventChannelFinder Interface

• list_channels• retrieve_channel


Origin

eventAccess

SKELETON

CODE

IfEvent Service Example

STUB

CODE

RUNTIME

LIBRARY

CLIENT

RUNTIME

LIBRARY

Event Service

NamingService

eventFinder eventFinderquery_events

eventAccessget_origin

Origin


IDL: Network Service

• Interfaces• NetworkExplorer• NetworkFinder• ConcreteNetworkAccess


IfNetwork: NetworkExplorer Interface

• locate_channels• locate_by_sampling• locate_all • retrieve_channel• retrieve_grouping


IfNetwork: NetworkFinder Interface

• retrieve_by_id• retrieve_by_code• retrieve_by_name• retrieve_all


IfNetwork: ConcreteNetworkAccess

• get_attributes• retrieve_grouping• retrieve_groupings• retrieve_channel• retrieve_channels_by_code

• locate_channels• retrieve_instrumentation• retrieve_calibrations• retrieve_time_corrections• retrieve_all_channels


IfSeismogramDC Service

• Interfaces• DataCenter• DataCenterCallBack


IfSeismogramDC: DataCenter Interface

• available_data• retrieve_seismograms• queue_seismograms• retrieve_queue• cancel_request• request_status• request_seismograms


IfSeismogramDC : DataCenterCallBack Interface

• return_seismograms• return_group• return_error• finished• canceled


IDL IfNetwork Service Indepth

• Defines Network Service Objects• Attributes of each object• Methods of each object


IDL Network Service

#include "Fissures.idl"module Fissures { module IfNetwork { //******************************************************** // Data Types //********************************************************* typedef string StationCode; //5 char typedef string SiteCode; //2 char typedef string ChannelCode; //3 char typedef string NetworkCode; //2 char

struct NetworkId { NetworkCode network_code; Time begin_time; };


IDL Network Service, cont 1 struct ChannelId { NetworkId network_id; StationCode station_code; SiteCode site_code; ChannelCode channel_code; Time begin_time; }; typedef sequence<ChannelId> ChannelIdSeq;

valuetype Channel { private ChannelId id; public string name; public Orientation an_orientation; public Sampling sampling_info; public TimeRange effective_time; public Site my_site; ChannelId get_id(); ChannelCode get_code(); }; typedef sequence<Channel> ChannelSeq;


IDL Network Service, cont 2 //************************************************************ // Exceptions //************************************************************ exception ChannelNotFound { ChannelId channel; };

//************************************************************ // Iterator //************************************************************ interface ChannelIdIter { long how_many_remain(); boolean next_n( in long how_many, out ChannelIdSeq seq ); void destroy(); };


IDL Network Service, cont 3 //************************************************************ // NetworkDC Interface //************************************************************ interface NetworkDC { readonly attribute NetworkExplorer a_explorer; readonly attribute NetworkFinder a_finder; }; //************************************************************ // NetworkExplorer Interface //************************************************************ interface NetworkExplorer { ChannelIdSeq locate_channels( in Area the_area, in SamplingRange sampling, in OrientationRange orientation, in long seq_max, out ChannelIdIter iter ); Channel retrieve_channel( in ChannelId a_channel) raises ( ChannelNotFound ); }; };};


IDL compile

jidl --impl IfNetwork.idl

Channel.java Channel_impl.javaChannelCodeHelper.javaChannelHolder.javaChannelHelper.javaChannelId.javaChannelIdHelper.java ChannelIdHolder.javaChannelIdIter.javaChannelIdIterHelper.javaChannelIdIterHolder.javaChannelIdIterOperations.javaChannelIdIterPOA.java ChannelIdIter_impl.java

NetworkIdHelper.javaNetworkIdHolder.javaOrientationRange.javaOrientationRangeHelper.javaChannelHolder.javaOrientationRangeHolder.javaOrientationRangeValueFactory_impl.javaOrientationRange_impl.javaSamplingRange.javaSamplingRangeHelper.javaSamplingRangeHolder.javaSamplingRangeValueFactory_impl.javaSamplingRange_impl.javaSite.java


Overview



Sample Client

Sample Java Client to retrieve channel informationExplorerRetrieveChannel.java:

import edu.iris.Fissures.*;import edu.iris.Fissures.IfNetwork.*; import edu.sc.seis.fissuresUtil.namingService.*;

public class ExplorerRetrieveChannel{


Sample Client, cont 1

public static void main(String args[]){ java.util.Properties props = System.getProperties(); if( args[1].equals("-props") ) { System.out.println("Opening prop file "+ args[2] ); java.io.InputStream in = new java.io.BufferedInputStream( new java.io.FileInputStream(args[2])); props.load(in); } ExplorerRetrieveChannel erc = new ExplorerRetrieveChannel(); org.omg.CORBA.ORB orb = org.omg.CORBA.ORB.init( args, props ); erc.run( orb );}



public int run ( org.omg.CORBA_2_3.ORB orb ) throws org.omg.CORBA.UserException { //======================================== // register factories for valuetype variables //======================================== edu.iris.Fissures.model.AllVTFactory vt = new edu.iris.Fissures.model.AllVTFactory(); vt.register(orb);

//======================================== // Get the NetworkExplorer //======================================== FissuresNamingServiceImpl fissuresNamingService = new FissuresNamingServiceImpl(orb); NetworkDC networkDC = fissuresNamingService.getNetworkDC( "edu/iris/dmc",”IRIS_NetworkDC”); NetworkExplorer networkExplorer = networkDC.a_explorer();



//========================================// Initialize some query structures//======================================== edu.iris.Fissures.IfNetwork.NetworkId networkId = new edu.iris.Fissures.IfNetwork.NetworkId("IU", new edu.iris.Fissures.Time( "2001-07-01T10:11:12.0000Z",-1));

edu.iris.Fissures.IfNetwork.ChannelId chanid1 = new edu.iris.Fissures.IfNetwork.ChannelId(networkId, "ANMO", "00” ,"BH1", new edu.iris.Fissures.Time( "2001-07-01T10:11:12.0000Z",-1));



//===========================================// Call retrieve_channel//===========================================try { Channel channel = networkExplorer.retrieve_channel( chanid1 ); printChannel( channel ); System.out.println(“Client finished”);} catch( ChannelNotFound ex ) { System.out.println("Channel not found" ); System.exit(-1); }} // End method run} // End of ExplorerRetrieveChannel


Sample Client, compile and run

setenv CLASSPATH FissuresIDL1.0.jarsetenv CLASSPATH ${CLASSPATH}:fissuresUtil1.0.jarsetenv CLASSPATH ${CLASSPATH}:FissuresImpl1.0.jarsetenv CLASSPATH ${CLASSPATH}:OB.jarsetenv CLASSPATH ${CLASSPATH}:OBNaming.jarsetenv CLASSPATH ${CLASSPATH}:log4j-1.2beta4.jarsetenv CLASSPATH ${CLASSPATH}:.

javac ExplorerRetrieveChanneljava ExplorerRetrieveChannel -props prop.dmc


Sample Client, output

****************************networkExplorer.retrieve_channel****************************Channel: IU (19880101000000.0000GMT) Station: ANMOLocation: 00Channel: BH1Name: IU.ANMO.00.BH1.2000-10-19,16:00:00.2002-11-19,21:07:00Azimuth: 89.0Dip: 0.0Starttime: 20001019160000.0000GMTEndtime: 20021119210700.0000GMTClient finished


Sample Client, property file

org.omg.CORBA.ORBClass=com.ooc.CORBA.ORBorg.omg.CORBA.ORBSingletonClass=com.ooc.CORBA.ORBSingleton## get the name serivce# The format for a corbaloc for a naming service is# corbaloc:iiop:host.domain:port/NameService# You should put your own choices for host.domain and port##ooc.orb.service.NameService=corbaloc:iiop:piglet.seis.sc.edu:7771/NameService

ooc.orb.service.NameService=corbaloc:iiop:dmc.iris.washington.edu:6371/NameServiceIfNetwork.domain=edu/iris/dmcIfNetwork.servername=IRIS_NetworkDC


Interfaces:Leveraging CORBA

WaveformService

NetworkServices

EventService

Researcher

IRIS

DM

CN

CED

C

GEO

SC

OPE

OR

FEU

S

Beijing DC

OHP DC


DHI Clients


More Information

[email protected]@iris.washington.edu

Mailing Lists:

Web Sites:Server, client and mailling list information:http://www.iris.edu/DHI

Fissures IDL and information:http://www.seis.sc.edu/software/Fissures

CORBA information:http://www.corba.org


Fissures Prototype Team - IDL

Philip Crotwell, University of South CarolinaTom Owen, University of South Carolina Paul Friberg, ISTISid Hellman, ISTITom McSweeney, IRIS DMCJohn Winchester, IRIS DMCMitchel Sanders A2B


DHI Server Developers

Chris Laughbon:Sandy Stromme:Sue Schoch:

Waveform serversEvent serverNetwork server


End of Talk

sep 22, 2003 data handling interface enhancing capabilities at the iris dmc iris data management...

Documents