5/21/2004 fred branski may 21, 2004 noaa/nws telecommunication gateway gts system

5/21/2004

Fred Branski May 21, 2004

NOAA/NWS Telecommunication Gateway

GTS System

5/21/2004

National Weather Service Telecommunication Gateway

The Primary Data Communications Switching Facility of the NWS

Operated 24 X 7 by the Telecommunication Operations Center Which is Part of the Office

of the Chief Information Officer

Housed in a Highly Resilient Facility with Dedicated Power and

Environmental Systems

5/21/2004


Serves the U.S. Public, Government and Commercial Hydrometeorological, Oceanographic, Climatological and

Emergency Management Communities

Critical Acquirer and Provider of Data and Products to and from Many Systems and Agencies

5/21/2004


The Advanced Weather Interactive Processing System (AWIPS)

and NOAAPortNetwork Control Facility (NCF)

The ASOS Operations and Monitoring Center (AOMC)

Aggressively Pursuing an Alternate Operational Facility

5/21/2004


ICAO - The Communications Portion of the Washington World Area Forecast System

(WAFS) Center

Aeronautical Fixed Telecommunication Network (AFTN) Switch

OPMET Data Bank Server

Critical to the Aviation Community for Acquiring and Providing Timely

Data and Products

5/21/2004


WMO - The Communications Portion of the Washington World Meteorological Center

Regional Telecommunication Hub (RTH)for WMO Region IV

Provides the Path for U.S. Meteorological and Related Products to the Rest of the World and

for their Products Back into the U.S.

5/21/2004 7

NWSTG Structure & FunctionNWSTG Structure & Function

Components

• 3 Mainframes – Active, standby, test & development

• >70 Unix servers (HTTP, FTP, File server, communications, data processing)

• >2 terabytes of online storage

• >160 point-to-point communication circuits

• 7 internal networks

5/21/2004 8


5/21/2004 9


Message Processing• Data input from various sources• Data is identified by WMO or AWIPS heading• Data stored in a receive queue along with

information retrieved from a switching directory (>130,000 entries)

• Some data selected for processing which may generate new products

• Data written to transmit queue(s) based on switching directory information

• Data sent to destination(s)

5/21/2004 10

NWSTG Structure & FunctionNWSTG Structure & FunctionMessage Processing

>850,000 messages received daily

>8.5 billion bytes received daily

5/21/2004 11

WMO Global Telecommunication System

Main Trunk Network

Bracknell Toulouse

Sofia

PragueOffenbach

Brasilia

Buenos Aires

Dakar Algiers

Cairo

Nairobi

Beijing

Jeddah

New Delhi

Tokyo

Washington

Moscow

Melbourne

Exeter Toulouse

Sofia

PragueOffenbach

Brasilia

Buenos Aires

Dakar Algiers

Cairo

Nairobi

Beijing

Jeddah

New Delhi

Tokyo

Washington

Moscow

Melbourne

The Improved MTN (IMTN)The Improved MTN (IMTN)The planned IMTN configuration should be achieved in 2004

Exeter

Sofia

Cloud IICloud II

Melbourne

Buenos Aires

TokyoBeijing

New Delhi

Nairobi

Cairo

Toulouse

Dakar

Algiers

MoscowWashington

Brasilia

Offenbach

Jeddah

Prague

Frame RelayFrame Relay by BT Igniteby BT Ignite

Frame RelayFrame Relay by Equantby Equant

Cloud ICloud I

5/21/2004 13

0

1

2

3

4

00 03 06 09 12 15 18 21 [UTC]

BINAN

[Mbytes/20 min] Tokyo to Washington (CIR=32kbps) (3 October 2003)

0

10

20

30

40

50

60

00 03 06 09 12 15 18 21 [UTC]

File

FAXBIN

AN

[Mbytes/20 min] Washington to Tokyo (CIR=768kbps)

IMTN Cloud I Frame

Relay

Melbourne

1.5Mbps

256kbps

Bracknell

256kbps

Access circuit 1.5Mbps

32kbps

16kbps 16kbps

16kbps

CIR=768kbps 32kbps

64kbps32kbps

32kbps

32kbps

WashingtonTokyo

FAX BIN AN

1

2

3

4

000 03 06 09 12 15 18 21 [UTC]

[Mbytes/20min] Tokyo to Melbourne (16kbps) (3 October 2003)

FileFAXBIN AN

4[Mbytes/20min] Melbourne to Tokyo (326kbps) (3

October 2003)

3

2

1

000 03 06 09 12 15 18 21 [UTC]

MTN Connectivity in RAII &V MTN Connectivity in RAII &V (slide from (slide from Hiroyuki ICHIJO (JMA)Hiroyuki ICHIJO (JMA)

Specific characteristics of IMTN data-Specific characteristics of IMTN data-communication cloudscommunication clouds

Cloud I Frame Relay

Melbourne

Tokyo

1.5Mbps

Washington

256kbps

Bracknell

256kbps

Access circuit 1.5Mbps

32kbps

16kbps 16kbps

16kbps

CIR= 768kbps 32kbps

64kbps32kbps

32kbps

32kbps

PVC

Flexibility of establishing logical connections (PVC)Asymmetric bandwidths (CIR: Committed Information Rate)

5/21/2004

Observations$ Over 450,000 Observations Distributed Daily

$ 152,000 Surface Observations

$ 3,100 Upper Air Observations

$ 4,700 Ship Reports

$ 46,000 Aircraft Reports

$ 48,000 DCP Reports

$ 200,000 Satellite Reports

$ Observations from 33,000 Places

5/21/2004

Model Output $ Over 96,000 Model Products Distributed Daily

$ 9 NCEP Models Run Daily

$Hurricane, Volcanic Ash and Hydrological Dispersion Model as Needed$EPA Air Quality Forecast

$ Also from Canada, United Kingdom, Japan, Australia, NWS River Forecast Centers, DoD and ECMWF

5/21/2004

Text Products

$ Over 36,000 Text Products Distributed Daily

$ 23,800 NWS Produced Text Products

$ 7,000 other U.S. Agency Text Products

$ 5,300 International Text Products

5/21/2004

Pictorial Products

$ Over 2,000 Pictorial Products Distributed Daily$ Most in Red Book Graphic Format

$OFCM Standard FCM-S2

$ CCITT T4 Fax and GIF products also

5/21/2004 19

Data Input to the NWSTGData Input to the NWSTG

• NWS (ASOS, WFOs, NCEP National Centers) – observations, guidance, forecasts, watches & warnings, national products

• FAA (ASOS) – aviation observations & intnl data• DoD (FNMOC, AFWA)• DoT (Coast Guard) – marine reports• Other Government and Scientific Sources (NOS,

OAR, USDA, Scripps Institute, SeaKeepers, etc.) – observations

• FEMA, DoA, EPA• ICAO (ISCS, AFTN) – aviation data• WMO (GTS MTN, Other GTS) – meteorological and

hydrological data• Commercial community

5/21/2004 20


Data Input Methods

• IP Sockets

• X.25

• Asynchronous

• FTP – http://weather.gov/tg/ftpingest.html

• Email – http://weather.gov/tg/emailingest.html

• Web - http://weather.gov/tg/bullguid.html

http://weather.gov/tg/ftpingest.html





http://weather.gov/tg/emailingest.html





http://weather.gov/tg/bullguid.html





5/21/2004 21


5/21/2004 22


5/21/2004 23

NWSTG Data Dissemination - PushNWSTG Data Dissemination - Push

Broadcast, multicast, and point-to-point data dissemination:

• NOAAPORT

• NWWS - NOAA Weather Wire Service

• EMWIN – Emergency Manager’s Weather Information Network

• ISCS – International Satellite Communications System

• FOS – Family of Services

5/21/2004 24


ISCS – International Satellite Communications System

• Supports two data dissemination requirements for

– ICAO World Area Forecast System (WAFS)

– WMO Regional Meteorological Telecommunication Network for Region IV (North America) supporting two-way communication (RMTN/GTS)

• C-band satellite coverage for– US area of responsibility - 30°W to 120°E

– European area of responsibility - 100°E to 30°W (SADIS)

5/21/2004 25


ISCS – International Satellite Communications

• Data includes aviation weather and other meteorological, hydrological, and climatological products

• Six data streams– WAFS Text, GRIB, and Facsimile– RMTN Text, GRIB, and Facsimile

• Volume: >630MB/day >180,000 Products/day• More Information at

– http://weather.gov/tg/iscscvr.html– http://www.nws.noaa.gov/iscs/

http://weather.gov/tg/iscscvr.html

http://www.nws.noaa.gov/iscs/



5/21/2004 26

NWSTG Data Dissemination - PushNWSTG Data Dissemination - PushInternational Satellite Communications System

ATLANTIC OCEAN REGION (AOR)ATLANTIC OCEAN REGION (AOR)

The AOR portion of the successor ISCS will be centered around the WorldCom shared Master Satellite Hub, in Andover, Maine.

The Master Satellite Hub will be linked via a 64 Kbps (128Kbps bursting) terrestrial (redundant) circuit to the National Weather Service Telecommunications Gateway (NWSTG) in Silver Spring, Maryland.

The NWSTG will continue to serve as the central point of data entry into the ISCS.

All ISCS data originating at the NWSTG will be transmitted, using TCP/IP network protocol, over to the WorldCom Andover facility for broadcast to all ISCS AOR sites.

5/21/2004 28

NWSTG Data Dissemination - PushNWSTG Data Dissemination - PushInternational Satellite Communications System

RMTN two-way sites shown in orange

ISCS WAFS – Pacific Ocean Region

5/21/2004 35


EMWIN – Emergency Managers Weather Information Network

• Hemispheric direct broadcast using GOES East/West

• Data includes a watches & warnings, forecasts, graphics, satellite imagery

• Volume: >85MB/day

• Used to distribute data to Emergency Managers

• More Information at– http://iwin.nws.noaa.gov/emwin/index.htm

http://iwin.nws.noaa.gov/emwin/index.htm






5/21/2004 36

NWSTG Data Dissemination - PushNWSTG Data Dissemination - PushEMWIN – Satellite Footprint

5/21/2004 37

EMWIN ArchitectureEMWIN Architecture

5/21/2004 38


NOAAPORT • AWIPS satellite broadcast network (SBN)

• C-band satellite covering the entire US

• Data includes NCEP model products, satellite imagery, observations, forecasts, watches & warnings

• Volume: >23 GB/day >2,000,000 Products/day

• Used to distribute data to– NWS field offices (WFOs and RFCs)

– Commercial Users with NOAAPORT Receiver Systems

• More Information at http://www.nws.noaa.gov/noaaport/html/noaaport.shtml

http://www.nws.noaa.gov/noaaport/html/noaaport.shtml






5/21/2004 39


AWIPS Network ControlFacility (ANCF)

NOAAPORT (SBN)3.5 T1 Circuits:

NCEP/NWSTG Data

GOES East

GOES West

Non-GOES Imagery/DCP

140+AWIPS Field Sites

NOAAPORT Receive

System (NRS)

Weather andForecastProducts

Satellite Imagery

Americom 4C-Band

MasterGround Station(MGS)

4 T1 Circuits

Source: NGIT AWIPS Briefing, L. Klet

5/21/2004 40


NWWS – NOAA Weather Wire Service• Nationwide satellite distribution system for text products

• C-band and Ku-band satellite covering the entire US

• Data uplinked from 14 WFOs/RFCs, 6 National Centers

• Data includes forecasts, watches & warnings

• Volume: >8 GB/day

• Used to distribute data to– Media

– Emergency Managers

• More Information at– http://www.nws.noaa.gov/nwws/

– http://dynis.dyncsc.com/contracts/nwws/index.html

http://www.nws.noaa.gov/nwws/

http://dynis.dyncsc.com/contracts/nwws/index.html







5/21/2004 41

NWSTG Data Dissemination - PushNWSTG Data Dissemination - PushNWWS – System Data Flow

Source: Detailed NWWS Data Collection Flowchart

5/21/2004 42


FOS – Family of Services (Push)

• Point-to-point and multicast communication services

• Serves commercial customers on a cost recovery basis

• More information at– http://www.nws.noaa.gov/datamgmt/fos/fosindex.html

• FOS OfferingsPublic Product Service

Domestic Data Service

International Data Service

High Resolution Data Service

Radar Product Service

Server Access Service (not push)

http://www.nws.noaa.gov/datamgmt/fos/fosindex.html

5/21/2004 43


Products Provided via Push• Content managed by the Data Management

group in the Operations Support and Performance Branch

• Changes made on a daily basis in response to operational requirements and customer requests

• Products identified in the NWSTG (the Switching Directory) listed at http://weather.noaa.gov/tg/wmohdg.shtml

• Products pushed via FOS, ISCS, NOAAPORT, and EMWIN listed at http://weather.noaa.gov/tg/dir_subset.shtml

http://weather.noaa.gov/tg/wmohdg.shtml

http://weather.noaa.gov/tg/dir_subset.shtml




5/21/2004 44

NWSTG Data Dissemination - PullNWSTG Data Dissemination - Pull

Servers provide access to stored data through various server protocols

• FTP – Access to data files via anonymous FTP at ftp://tgftp.nws.noaa.gov (since 1993!)

• FTPMail – Email based FTP for clients without the ability to establish interactive sessions

• HTTP – Access to data files via http://weather.noaa.gov/pub (since 1994!)

• FOS Server Access Services – FTP access through a dedicated connection

ftp://tgftp.nws.noaa.gov/

http://weather.noaa.gov/pub

5/21/2004 45


FTP – Access to data files via anonymous FTP at ftp://tgftp.nws.noaa.gov including

• NCEP model output http://weather.gov/tg/modfiles.html

• Product files (multiple messages or bulletins per file)http://weather.gov/tg/txtfiles.htmlhttp://weather.gov/tg/obsfiles.html

• WSR-88D Level III productshttp://weather.gov/tg/radfiles.html

• Satellite productshttp://weather.gov/tg/satfiles.html

• Facsimile products http://weather.noaa.gov/fax/index.html

See http://weather.gov/tg/dataprod.html for more information


http://weather.gov/tg/modfiles.html





http://weather.gov/tg/txtfiles.html

http://weather.gov/tg/obsfiles.html

http://weather.gov/tg/radfiles.html

http://weather.gov/tg/satfiles.html

http://weather.noaa.gov/fax/index.html

http://weather.gov/tg/dataprod.html

5/21/2004 46


FTPMail – Email based FTP

• Access to all FTPable products on ftp://tgftp.nws.noaa.gov

• Email an FTP command sequence to [email protected]

• Used primarily in the maritime community to retrieve products via satellite email services

• See http://weather.gov/tg/ftpmail.html for usage details


mailto:[email protected]

http://weather.gov/tg/ftpmail.html

5/21/2004 47


HTTP – Access to data files• Access all the products at

ftp://tgftp.nws.noaa.gov without FTP overhead at http://weather.noaa.gov/pub

• Provides more browsable alternative to FTP• Application-based HTTP file transfers are often

faster than FTP

• Traditional interactive Web services include– http://weather.gov– http://weather.noaa.gov– http://iwin.nws.noaa.gov


http://weather.noaa.gov/pub

http://weather.gov/

http://weather.noaa.gov/

http://iwin.nws.noaa.gov/



5/21/2004 48


FOS Server Access Services (SAS) – FTP access through a dedicated connection

• SAS customers establish a point-to-point connection with and NWS router

• Customer router hosting at the NWSTG for balanced multiple-circuit connections

• 64Kb, T1, multiple-T1, and T3 connections

• Access to a non-public anonymous FTP server holding the same data as ftp://tgftp.nws.noaa.gov


5/21/2004 49


File Formats

• Single product files: http://weather.noaa.gov/pub/data/raw/wg/wghw70.phfo.ffs.hi.txt

• Multiple product files: http://weather.noaa.gov/pub/SL.us008001/DF.an/DC.sflnd/DS.metar/sn.0021.txt

See http://weather.gov/tg/fstandrd.html for a description of how products are delimited within a file

• Non-headed product files: http://weather.noaa.gov/pub/SL.us008001/ST.opnl/MT.eta_CY.00/RD.20040108/

PT.grid_DF.bb/fh.0000_tl.press

• CWA-specific, Zone-specific or named files: http://weather.noaa.gov/pub/data/forecasts/zone/wa/waz007.txt http://weather.noaa.gov/pub/data/records/wa/seattle.txt

5/21/2004 50


Directory Structures• Descriptive directory naming:

http://weather.noaa.gov/pub/data/forecasts/zone/wa/waz007.txt http://weather.noaa.gov/pub/data/records/wa/seattle.txt

See http://weather.gov/tg/engfiles.html• Encoded Directory naming:

http://weather.noaa.gov/pub/SL.us008001/DF.an/DC.sflnd/DS.metar/sn.0021.txt http://weather.noaa.gov/pub/SL.us008001/ST.opnl/MT.eta_CY.00/RD.20040108/PT.grid_DF.bb/fh.0000_tl.press

See the Directory and File Naming Standards document at http://www.nws.noaa.gov/datamgmt/filstnd.html to understand how to decode

5/21/2004 51


Directory Structures – decoding examplehttp://weather.noaa.gov/pub/SL.us008001/DF.an/DC.sflnd/DS.metar/sn.0021.txt

SL.us008001 – server location: United States (us), NWSTG (008), Server 1 (01)

DF.af – data format: WMO bulletins of alphanumeric data as GTS exchanged (ASCII)

DC.sflnd – data category: Surface data (land)

DS.metar – data subcategory: Aviation – METAR

sn.0021.txt – sequence number file (21st in the sequence) containing text data

5/21/2004 52


Directory Structures – decoding examplehttp://weather.noaa.gov/pub/SL.us008001/ST.opnl/MT.eta_CY.00/

RD.20040108/PT.grid_DF.bb/fh.0000_tl.press

SL.us008001 – server location: United States (us), NWSTG (008), Server 1 (01)

ST.opnl – status of data files: operational

MT.eta – model type: Eta Forecast Model

CY.00 – cycle of run: 00 UTC

RD.20040108 – reference date: January 08, 2004

PT.grid – product type: gridded

DF.bb – data format: WMO bulletins of binary fields in GRIB as GTS exchanged

fh.0000 – forecast hour 0000 UTC

tl.press – type of level or layer: pressure

5/21/2004

KWBC Weather BureauCommunications

Began as a manual data collection and distribution center in World War II

1954 - 9 Teletype Circuits1966 - 2 IBM 360 Computers 104 Ckts, one at 3.6k bps1992 - 3 HDS 9060 Computers T1 to NMC (NCEP)1998 - 2 IBM 9672, 1 IBM 9673 2 IBM 6000/F50 servers 2 IBM 6000/H50 servers 10 Mbs Ethernet to NCEP ATM and multiple T1s

Today?

5/21/2004

A View of the Past

5/21/2004 55

Your Data and Product AdvocatesYour Data and Product AdvocatesData Management StaffData Management Staff

P Fred Branski - Team Leader, Data [email protected] (301) 713-0864 ext 146P Julie Hayes - Family of Services Manager [email protected] (301) 713-0864 ext 120P Walter Smith - Senior Data [email protected] (301) 713-0864 ext 139P Richard Robinson - Data Manager [email protected] (301) 713-0864 ext 179P Sharon Abbas – Data Manager [email protected] (301) 713-0864 ext 120P Tom King – Statistics Analysis and Support [email protected] (301) 713-0864 ext 110

P KWBC Communication Control Center (CCC) - Tech ControlPoint of contact 24 hours every day Phone: (301) 713-0902 Fax: (301) 587-1773

5/21/2004


GTS System

Questions / Discussion

5/21/2004 fred branski may 21, 2004 noaa/nws telecommunication gateway gts system

Documents

timely data

provider of data

entriessome data

communications portion

switching directory

resilient facility

related products

chief information officer