nws office of science and technology systems engineering center (skjei telecom) mike asmussen 1
TRANSCRIPT
NWS Office of Science and TechnologySystems Engineering Center
(Skjei Telecom)
Mike Asmussen
1
Outline• Introduction• Requirements Development
– Requirements approach– Key documents– Requirements summary
• Generalized IT architecture– Assumptions– Architecture efforts to date– Wx Products– FAA IT Architecture– NWS / NOAA IT Architecture Summary
– Next Steps
2
3
Key Concepts• NWS / NOAA architecture will follow a System
of Systems approach• No one entity is building THE CUBE • It will be a federation of network-enabled
services (some existing services, and some yet to be deployed)
• Each service will be “registered” in a federated registry/repository, which • Will provide descriptive information about each
available service (via published metadata) and,• Will point any interested service user to the
appropriate service endpoint to allow service access4
5
Requirements Approach• Requirements Challenges
• Most available requirements are high level (not IT requirements)• No single document serves as the “definitive” requirements source• Some requirements not clearly characterized as:
• IOC/MOC/FOC• Single Authoritative Source (SAS) / non-SAS
• Approach• Surveyed numerous JPDO and NOAA documents• Determined relevant IT related requirements (as well as derived IT
requirements)• Categorized requirements• Created table of requirements
• Where possible, assigned timeframe of deployment and whether requirement is SAS-specific
• Identified Weak Areas• Only limited IT performance and security requirements have been developed
to date• IOC Cube contents are still under consideration (required use cases still
being examined)
6
Key Documents
7
Document Name Version Date SourceConcept of Operations for the Next Generation Air Transportation System V2.0 6/13/2007 JPDO
NextGen Network-Enabled Weather IT CONOPS 3.2 8/20/2008 NCAR, MITLL, NOAA/GSD
NextGen ATS Enterprise Architecture V2.0 6/22/2007 JPDO
Four-Dimensional Weather Functional Requirements for NextGen Air Traffic Management 0.1 1/18/2008
JPDO Functional Rqmts Study Team
Weather Concept of Operations V1.0 5/13/2006JPDO Weather Integrated Product Team
NextGen Weather Plan 0.6 3/20/2009 JPDOList of IOC and FOC products that NWS has committed to provide for NextGen
Final Performance Requirements (iFR) First Working Draft Wrapper - 4-D Weather Data Cube SAS Draft 2/11/2009 JDPONextGen Weather Information Database - Information Technology Needs (Draft SON) Draft 3/13/2009 OSTConcept of Operations and Operational Requirements - WIDB for the NextGen 07-042 5/4/2009
Office of Climate, Water and Weather Services
Definition of 4-D SAS 6/17/2009NEWP presentation by JPDO Wx Policy Team2
ATM Wx Integration PlanDraft V0.7 4/22/2009 JPDO
Requirements Summary• Access
– Aggregate overlapping requests
– High BW and low BW access methods support
• Agility– Ability to add new systems,
services, products, data fields, users, etc. w/o system interruptions
– Support legacy and new systems together
– Scalable over time• Archival / logging
– Past transaction retrieval (e.g., for accident investigations)
Availability Fault tolerance Load balancing Essential FAA service support -
.999 Critical FAA service support
- .99999 SAS (essential):
MTTR - 0.5 hr, MTBF – 5,000 hrs Outage max – 10 mins
SAS (critical): MTTR - 0.5 hr MTBF – 50,000 hrs Outage max – 6 secs
8
Requirements Summary (cont)• Compatibility
– With CWSUs, AWC, WFOs, Tower systems, TRACON systems, ARTCC, ATCSCC
– With FAA architecture• Contents
– Support Wx Products required for aviation purposes, for example:• NOAA provided, FAA-provided, 3rd party
provided• North American and global• Forecasts model data (probabilistic)• Sensor products
– Radar, lightning, satellite, aircraft sensors, airport, ground, ocean, air, METARs
• Observations – PIREPS• Advisories, watches, warnings• Climatological data
– Formats• Grid based (machine readable) where
possible (e.g., NETCDF4)• Encoded versions of legacy text products
(via WXXM or JMBL)• Otherwise, text and graphic? Binary?
Data Consistency Deconflicted SAS (temporal and
spatial) Data Formats / Protocols
Allowing ease of exchange Standards based (e.g. HTTP, XML,
SOAP, OGC) Discoverability
(metadata/registry/ repository) Products/data Formats Access methods Associated services
9
Requirements Summary (cont) Distributed (not centralized) Ease of integration with aviation
decision support tools Governance
Access control Standards Common ontologies Business rules SAS decisions
Intelligent processing Data interpolation
Netcentric SOA based System of systems Legacy system support (along
with new systems) QOS / Performance
Varies by user / use case / function Request management
Request / reply exchange mechanisms
Data for time period of choice Data for geographical construct of
choice Product / data field of choice Priority Desired format / translations Compression
10
Requirements Summary (cont)• Security
– Data security– Physical / network security– Field by field, product by
product access control• Subscription management support• SAS management and governance• System management
– Failure detection / switchover– Load balancing– Health monitoring / analysis /
trending / logging• Users
– Aviation (primarily), governmental, commercial, military, international
– Non-aviation, research, NOAA-internal
• Verification / quality control
11
12
AssumptionsThe following key assumptions are driving
NOAA’s NextGen IT Architecture:Use of a System of Systems approachCompatibility with key NOAA and NWS
Enterprise Architecture guidanceCompliance with NextGen requirements (with
flexibility to evolve as requirements evolve)Supports IT ConOps Use CasesCompatible with evolving FAA ArchitectureSupportive of NextGen Enterprise Architecture
definition of Business Services / Operational Activities
13
Architecture Efforts to Date• Compiling requirements / use cases• Identifying required Wx Cube data (and candidate source / destination systems), flows, and formats
• Reconciling IOC Product list , FAA Product Flows spreadsheet, SA Plan, others• Working closely with FAA Architecture team to ensure architecture compatibility
• FAA architecture approach – Hub and spoke / Store and forward– Federated registry / repository (for metadata)– FAA Origin Servers (ingest/store/serve FAA data)– FAA Distribution Servers (cache/handle requests of FAA users)– FAA External Distribution Servers (handle requests to and from systems externally to FAA)– Based on WCS/WFS Reference Implementation (RI) being developed by MITLL, NCAR, GSD which will be
available for NOAA re-use• Reconcile JMBL vs. WXXM
• Working with GSD to evaluate use of JMBL – several JMBL RIs may be available for NOAA use• Standards decisions
– Non-gridded– JMBL vs. WXXM (or both) for NOAA (FAA is standardizing on WXXM)
– Gridded data formats – NetCDF4/HDF5, GRIB2? Others?
– Web Services / message exchange standards (e.g, WCS, WFS, JMBL, SOAP, XML, WMS? etc)– Metadata standards
• Beginning to work with NOAA/NWS weather system owners to ensure compatibility of IT architecture / migration approach
• NWS IT System Architecture document and IT System Design document to be developed and released by March 2010 (with interim drafts prior)
14
Sample Wx Products List for NextGen Inclusion NEXRAD Level III Lightning data MADIS (TBD subsets) GOES data METSAT data TAFs METARs/SPECIs SIGMETS / G-SIGMETS AIRMETS / G-AIRMETS Surface fronts Meso-scale boundaries 3-D reflectivity products CCFPs CWAs
• MISs• LAMP output• Tropical cyclone
bulletins• FAs• Tornado watch /
warnings• Severe thunderstorm
watch / warning• Convective outlook• Non-convective
watches, warnings, advisories
• Freezing level analysis• Surface analysis• High level SIG Wx• Verification statistics
• Mid level SIG Wx• Low level SIG Wx• Winds aloft• RUC outputs• WRF-RR outputs• HRRR outputs• NCWF• NCWD• GTG• FIP• CIP• Global wind grids• ASOS OMOs• Others?
15
Resultant Potential Candidate Systems for Cube Inclusion ADDS AWIPS CAWS GIS system IOOS IRIS Lightning Data Services MADIS MDCRS NCDC NCEP/NOMADS
GFS (Global Forecast System) HRRR LAMP NAM (North American
Mesoscale) RUC (Rapid Update Cycle) WRF-RR
• NDFD• NDGD
• NESDIS• GAS (GOES R)• NDE (NPOESS)• Non-NOAA systems
• NEVS (RTVS, Stats on Demand)• NEXRAD (Radar Data Server)• NSSL Mosaics• RIDGE radar servers• TOC (NWSTG/NOAANET)• Web Farms• Other Sources for?
• Canadian Radar• Puerto Rico model data• STMAS data• UKMET model data 16
17
Layered Architecture Approach
18
Internal FAA Architecture Concept
End Users
ConsumerCube
Service Adaptor(CCSA)
Origin Servers(with
System Ingest Adaptor
and Provider Data)
DistributionServersIP
Network
ProviderSystem
4-D Wx Data Cube
Registry/Repository
ConsumerSystems Net-
EnabledServices
19
Internal FAA Architecture
20
Weather Data Clients
FTI WAN / SWIM
Weather DataOrigin Server
Router
Weather DataOrigin Server
Router
Router
Weather Data Clients
DistributionServer
Registry/Repository
ED-8 Gateway
ITWS ADAS
Weather DataOrigin Server
Router
NWP
DistributionServer
DistributionServer
Consumer Cube Service Adaptors
Consumer Systems
Consumer Cube Service Adaptors
Weather Data Clients
Consumer Cube Service Adaptors
Distribution Servers for
External Access
Consumer Systems
Consumer Systems
DMZ
21
22
23
Definitions Cube Input Edge Server (CIES)
Allows remote access to the weather data (or subsets thereof) via WCS/WFS/other web services. Provides for the ingest of weather data required by the Cube (obtained either directly from the native
source or via a Service Adaptor – which is mostly likely case for most data sources initially) Performs the necessary processing and local storage
Cube Output Edge Server (COES) Provides for the request and retrieval of Cube data from remote WCS/WFS/other web services Performs the necessary processing and local storage Allows access to the data by the requesting local destination system.
Service Adaptor (SA) Source Service Adaptor (SSA) - Performs processing to:
Transform native or legacy source wx data that is required for publishing to the Wx Cube into a format appropriate for ease of access via Cube Input Edge Servers (e.g., transformed into one of several supported standards)
To make source wx data available to Cube Input Edge Servers via a convenient and reliable network accessible means (e.g., Web Services-based communication) where such a means may not currently exist.
Destination Service Adaptor (DSA) - Performs processing to: Transform wx data from a format appropriate for ease of access by Cube Output Edge Servers into a
native or legacy format compatible with the destination system A Service Adaptor may be optional, where all such SA processing may instead be performed directly by a
Cube Input Edge Server or Cube Output Edge Server or internal to the source or destination system themselves (for future systems).
Optional Accessible Storage Optional storage to temporally hold wx data before being ingested into the cube or before being processed
for use by the requesting destination system
24
25
Weather Data Clients
FTI WAN / SWIM
Weather DataOrigin Server
Router
Weather DataOrigin Server
Router
Router
Weather Data Clients
DistributionServer
Registry/Repository
Registry/Repository
ED-8 Gateway
Federated
AWIPS
MADIS
NOMADS
NDFD
NEVS
Lightning Data
Sat Data
Cube Input Edge Server
Cube Input Edge Server
Cube Input Edge Server
Cube Input Edge Server
Cube Input Edge Server
Cube Input Edge Server
Cube Input Edge Server
Wx CubeData
Sources
AWIPSMADIS
Radar Data
Wx CubeData Destinations
Cube Output Edge Server
Cube Output Edge Server
Cube Output Edge Server
NWS IP NETWORKS (NOAANET)
NOAA/NWS FAA
CROSS ORGANIZATIONAL NOTIONAL ARCHITECTURE
ITWS ADAS
NWSTG
Weather DataOrigin Server
Router
NWP
DistributionServer
DistributionServer
Consumer Cube Service Adaptors
Consumer Systems
Consumer Cube Service Adaptors
Weather Data Clients
Consumer Cube Service Adaptors
Distribution Servers for
External Access
Consumer Systems
Consumer Systems
DMZ
Example Architecture Use Cases Assumptions
Reg/rep has already been queried in all cases Does not reflect any security aspects
Use Case examples: Requestor-FAA, Data Source – NOAA, Data Type – NonGridded (WFS) Requestor-FAA, Data Source – NOAA, Data Type – Gridded (WCS) Requestor-NOAA, Data Source – FAA, Data Type – NonGridded (WFS) Requestor-NOAA, Data Source – FAA, Data Type – Gridded (WCS) Requestor-NOAA, Data Source – NOAA, Data Type – JMBL Requestor-FAA, Data Source – NOAA, Data Type – NonGridded (JMBL) Requestor-FAA, Data Source – NOAA, Data Type – Gridded (JMBL)
26
27
28
29
30
31
32
33
Optional CIES-Accessible
Storage
Source Service Adaptor (SSA)
Wx Data Source System
Wx Data Source System
WX CUBE
NOAA CUBE SOURCE ALTERNATIVE APPROACHES
Intermediate Stand-alone SA
with CIES
Fully SOA Solution
(embedded CIES)
SOA via External
CIESWx Data Source System
Optional CIES-Accessible
Storage
Source Service Adaptor (SSA)
Wx Data Source System
Intermediate Bolt-on SA with CIES
Net enabled
IF
System Ingest
Adaptor
CIES processing /
storage
Net enabled
IF
System Ingest
Adaptor
CIES processing /
storage
Net enabled
IF
System Ingest
Adaptor
CIES processing /
storage
Net enabled
IF
System Ingest
Adaptor
CIES processing /
storage
CIES
CIES
CIES
CIES
Example Architecture Use Cases Assumptions
Reg/rep has already been updated in all cases Does not reflect any security aspects
Use Case examples: Loading data into Cube: Source System to CIES via SSA Loading data into Cube: Source System to CIES using Optional Storage Loading data into Cube: Source System to CIES Directly
34
35
36
37
38
NOAA CUBE DESTINATION ALTERNATIVE APPROACHES
Fully SOA Solution
(embedded COES)
Optional COES-Accessible
Storage
Destination Service Adaptor (DSA)
Wx Data Destination SystemNet
enabled IF
System Export
Adaptor
COES processing /
storage
Wx Data Destination
System
WX CUBE
SOA via External COES
Wx Data Destination System
Optional COES-Accessible
StorageWx Data Destination System
Intermediate Bolt-on SA with COES
Destination Service Adaptor (DSA)
Net enabled
IF
System Export
Adaptor
COES processing /
storage
Net enabled
IF
System Export
Adaptor
COES processing /
storage
Net enabled
IF
System Export
Adaptor
COES processing /
storage
Intermediate Stand-alone SA
with COES
Data Request
Data Request
Data Request
COES
COES
COES
COES
Example Architecture Use Cases Assumptions
Reg/rep has already been queried in all cases Does not reflect any security aspects
Use Case examples: NOAA Cube request via DSA (Intermediate Stand-alone or Bolt-on SA) NOAA Cube request with optional storage (Intermediate Stand-alone or
Bolt-on SA) NOAA Cube request from Destination System (SOA via External COES) NOAA Cube request directly to Data Source (Embedded COES)
39
40
41
Wx Cube Data
SourceCIES
Wx Cube Data
Source
Wx Cube Data
Source
Wx Cube Data
DestinationCOES
Wx Cube Data
Destination
Wx Cube Data
Destination
Centralized Options
42
Redundancy Options
CIES
Wx Cube Data
Source
CIES
CIES
COES
Wx Cube Data
Destination
COES
COES
43
44
Architecture Next Steps• Identifying definitive source for each Cube WX product• Defining Cube “format” for each Wx Product• Resolving JMBL/WXXM standards decision (or decision to support
both) and finalizing all exchange formats / protocols / standards• Involving stewards of each Cube candidate system in determining
details of “connecting” to the Cube• Refining requirements / use cases (performance, security,
verification, varied QOS offerings, external interfaces/system compatibilities, weather products needed)
• Determining key telecommunications infrastructure requirements, interfaces, and implementation details
• Finalizing high level NOAA Cube IT architecture (while ensuring compatibility with FAA architectural approach)
• Developing design for Cube edge servers
• And a million other things to do.
45