intersite coordination (isc) mark mathewson. sept 24-27, 2002intersite coordination2 overview...

Intersite Coordination (ISC)Intersite Coordination (ISC)

Mark Mathewson

Sept 24-27, 2002 Intersite Coordination 2

OverviewOverview

Motivation for ISC How ISC Works GFESuite ISC Capabilities

Including several lab exercises Diagnosing ISC Problems Discussion


Motivation for ISCMotivation for ISC


Motivation for ISCMotivation for ISC

Obvious – want your forecast to be consistent with your neighbors

But previously, it wasn’t obvious if your forecast was different. Hard to compare words

NDFD (National Digital Forecast Database) changes all of this.


Motivation Motivation for ISCfor ISC


How ISC WorksHow ISC Works


How ISC WorksHow ISC Works

Sending ISC Grids is triggered by the forecaster in a variety of ways.

Grids are sent over the WAN to your neighbors.

Grids are mosaic’d with other WFO ISC grids and stored in the ISC database.


ISC InternalsISC Internals


Spatial MosaicSpatial Mosaic


Temporal MosaicTemporal Mosaic


Sample ISCSample ISC Database Database

Central Cluster NDFD

EAX Perspective


Mosaic Technique Mosaic Technique CharacteristicsCharacteristics

There may be more than one ISC grid for one of your Fcst grids. Max/MinT across time zones are interesting.

There may be “blank” areas if not all adjacent WFO grids are available.


NDFD and Intersite CoordinationNDFD and Intersite Coordination Related – but not the same.

WFO

WFO

WFO

Fcst

Fcst

Fcst

WAN

WAN

WAN

Intersite Coordination Grids

Official

Official

Official

NDFD CentralServer

NDFD

Customers

WAN

WAN

WAN

NDFD Grids


GFESuite ISC CapabilitiesGFESuite ISC Capabilities


GFESuite ISC CapabilitiesGFESuite ISC Capabilities Sending Grids Displaying ISC Mosaic Displaying ISC and Fcst grids together Discrepancy Tools

IFPS10 ISC and IFPS11 ISC is different. Improved dialogs and use. IFPS12 has more changes, including ISC for

combined public and marine areas


Send ISC DialogSend ISC Dialog Select which weather elements Select time period -- allows single grid Select which sites to transmit your grids to


Publish DialogPublish Dialog

Can select weather element

Can select time period

Can choose to send ISC or not All sites always


Save Weather Element DialogSave Weather Element Dialog

Sends to all sites Sends just the grids that

are to be saved In IFPS12 In IFPS11, all grids

from weather element sent


sendISCsendISC

sendISC program can be used to send ISC grids. Can define: Time range over which grids are sent Weather Elements Sites, including sites that are NOT defined in

ISC_Sites. Official database or Fcst database


Displaying ISC GridsDisplaying ISC Grids

Load ISC database from Weather Element Browser


Lab Exercise 1Lab Exercise 1

Edit today’s Maximum Temperature grid. Send it to all your neighbors. View the ISC mosaic by loading the ISC

database through Weather Element Browser Watch as other sites appear/change Status bar indicates receipt of data Your data will also appear


GFESuite – “Show ISC Mode”GFESuite – “Show ISC Mode”

Available from the toolbar

Merges Fcst database over your CWA with ISC database over remainder of grid



Entire grid from EAX

OAX

EAX

SGF

TOP

ICT



Samples show: ISC Update Time ISC Site Identifier “P” for data from published database Data values

Markers show: Similar to samples, but no data values Useful for anchoring to monitor ISC



Data Value

Site IDUpdate Time

Official DB

SAMPLEs

MARKERs



Samples/Markers Site ID, UpdateTime, “Publish” flag can be

turned off Markers

Set are calculated with ifpServer edit areas on startup

Suggest you remove extra ones, move existing ones the way you like as “SITE”.


Grid Data HistoryGrid Data History


Lab Exercise - 2Lab Exercise - 2

Use tomorrow’s Maximum Temperature Modify it, choose whether to publish or just send

ISC Experiment with “Show ISC Mode”

Set up “markers” – MB3 with sample tool Explore turning on/off sample modes and marker

modes through Maps->Samples and Maps->Markers menus

Try Time-Stepping – observe behavior with/without “Show ISC Mode” on.


““Show ISC Mode” effect on Show ISC Mode” effect on toolstools

You already see the change in the Sample Tool.

When “Show ISC Mode” is on: Pickup value tool pickups Fcst or ISC data Move/Copy pickups Fcst or ISC data Contour Tool pickups Fcst or ISC data Pencil Tool pickups Fcst or ISC data


““Show ISC Mode”…Show ISC Mode”…

Smart Tools are not affected by ISC mode. If you want to access ISC data from smart

tools, “T” is always Fcst, no ISC data. Use getGrids() to access ISC data.

Temporal Editor – ignores ISC mode Grid Manager – doesn’t show ISC mode Spatial Editor Legends – do change



Experiment with “Show ISC Mode” on and off with: Sample Tool Move/Copy Tool Pencil Tool Contour Tool Smart Tools – such as Assign Value and

Smooth


Discrepancy ToolsDiscrepancy Tools

Discrepancy Tools are simply smart tools. You can extend and write your own.

Allow you to analyze all grids quickly. Highlights grids in grid manager if thresholds

exceeded Can look at difference grid Can look at edit area that exceeds thresholds Uses discrepancy value, or discrepancy grid.


Discrepancy ToolsDiscrepancy Tools

Procedure ISC_Discrepancies

Tools Show_ISC_Highlights Show_ISC_Grid Show_ISC_Area

Edit Area ISC_Tool_Area


ISC_Tool_AreaISC_Tool_Area Calculated when ifpServer starts. Tools use this area for calculations.

CWA boundary


Discrepancy AlgorithmsDiscrepancy Algorithms

Four neighboring points are examined

Discrepancy Value or Grid used to determine significant discrepancies

Uses “Composite” Grid (Fcst + ISC)

Neighboring Points


Discrepancy AlgorithmsDiscrepancy Algorithms ISC-Fcst Difference Subtracts two fields Discrepancy Value or Grid used to determine

significant discrepancies

Fcst ISC

EAXOAX EAXOAX


Discrepancy Value and GridDiscrepancy Value and Grid

Can set discrepancy value for each weather element.


Discrepancy GridDiscrepancy Grid Can create discrepancy grids.

Allows varying values per gridpoint.

Use normal editing tools to create it.


ISC_Discrepancies DialogISC_Discrepancies DialogTime Period

Weather Elements


Discrepancy HighlightingDiscrepancy HighlightingGrids highlighted in red indicate that thresholds were exceeded.


Difference GridDifference Grid

Original Fcst/ISC Grids

83

56

Difference Grid

Fcst – ISC grid = TDiff


Discrepancy Edit AreaDiscrepancy Edit Area

Edit Area, Exceeding thresholds



Edit MaxT, Save it Send grids Run ISC_Discrepancy tool

Grids are highlighted Look at discrepancy edit area Look at discrepancy difference grid Reset the highlights

Experiment with various options. Try both algorithms.


ISC ISC (current)(current) Limitations Limitations

Supplied tools don’t handle: Wind direction Weather

ISC area based on shapefiles CWAs only through IFPS11 Marine and CWAs IFPS12+


ISC ISC (current)(current) Limitations Limitations

Remember, ISC display and editing tools (non-smart), are hard-coded. Need your input to improve them.

However, there really are no other tool limitations, since smart tools are used. You can write your own algorithms/techniques.


Diagnosing ISC ProblemsDiagnosing ISC Problems


Diagnosing ISC ProblemsDiagnosing ISC Problems

ISC has a fairly complicated processing path.

Numerous log files are used to diagnose problems.

Log Files are documented.


Common ISC ProblemsCommon ISC Problems

I can’t see other site’s ISC grids.

I can’t see my own ISC grids.

Other sites can’t see my ISC grids.


Log Files – the key to diagnosing problemsLog Files – the key to diagnosing problems



Receiving Grids

for iscDataRec

17:13:52.991 EVENT: received: /data/adapt/GFESuite/iscDataRec/FSLC-339357.001 859317:13:53.165 EVENT: Sent to: rpp-fslc 98000000

for iscMosaic

00:20:00.348 iscMosaic.py 791 EVENT: iscMosaic Starting00:20:00.528 iscMosaic.py 665 EVENT: Processing file=/home/awipsusr/release/products/ISC/iscWO7Sgf00:20:00.533 iscMosaic.py 665 EVENT: Input file size: 275092iscMosaic.py 665 VERBOSE: Processing T_SFC #Grids= 23 Site= GID00:20:56.413 iscMosaic.py 791 EVENT: iscMosaic Finished

Traceback with problem if fails, starting in IFPS11



Sending Grids

for ifpnetCDF:

18:36:38.191 ifpnetCDF.py 918 EVENT: ifpnetCDF Starting18:36:38.734 ifpnetCDF.py 659 EVENT: Saved 10 T grids18:36:39.234 ifpnetCDF.py 659 EVENT: Saved 9 Td grids18:36:44.632 ifpnetCDF.py 983 EVENT: ifpnetCDF Finished

For iscExtract:

16:53:50.778 EVENT: Sent: size: 8594 ,wctime: 15.47704494 ,cputime: 0.14 ,sites: ['FSLC'] ,time: -s 19700101_0000 -e 20380119_0314 ,parms: ['T']

For iscd:

17:15:02.814 EVENT: sending: fname: @7955.1.isc.gz ,size: 8591 ,to: ['FSLC']


Collaboration TechniquesCollaboration Techniques

Is ISC really enough? Coordination of grids may come too late in the

forecast process. NDFD Clusters have been experimenting

with: FX/C or FX/Collaborate


DiscussionDiscussion


DiscussionDiscussion

Shortcomings of current tools/methods Tools which could help?

ISC Consensus Tool Methodology which could help?

intersite coordination (isc) mark mathewson. sept 24-27, 2002intersite coordination2 overview...

Documents

ifps11 isc

isc modesamples

isc modeavailable

wfo isc grids

isc modeentire grid

adjacent wfo grids

toolbarmerges fcst database

official database