an analytical approach to emergency communications exercises · •multiple injection points speed...

An Analytical Approach to Emergency Communications Exercises

Cascadia Rising

• June 8 and 9, 2016

• Scenario: A catastrophic 9.0 earthquake and tsunami impacting the Pacific Northwest.

• 6,000 participants drawn from local, state, federal agencies and NGOs.

• Major disruptions to commercial and government telecommunications infrastructure and the power grid anticipated.

• Situation reports from within the disaster area would be critical to developing a cohesive and efficient response.

1/30/2017 Radio Relay International 2

FEMA Requirements

• High Frequency methods would be FEMA’s “weapon of choice”– Survivable– Decentralized– Not reliant on distributed infrastructure– Immune to topographical problems

• All Radio Frequency – No reliance on Internet, PSTN or other distributed infrastructure.

• Mode neutral – maximum interoperability.

• Focus: Long-haul communications between simulated disaster area and FEMA National Response Coordinating Center (NRCC) at Washington, D.C. tested.


Selected System Characteristics

• Standardized procedures throughout North America.

• Universal message format provides superior accountability and network management tools including rules for message prioritization.

• Unified national-level management facilitates coordination across jurisdictional boundaries.

• Full interoperability across multiple modes and networks.

• Common denominator modes (universality) retained.

• Automated digital network available.

• Operator dispersal – decentralized.


Specific Areas of Interest

• Test reliability of the communications system to maintain connectivity across a broad range or RF conditions.

• Test the system’s prototype National Emergency Communications Response Plan.

• Examine the relative performance of the individual networks within the system:– Timeliness– Accuracy– Operator performance


System Structure

An overview of network types tested during the exercise.


Western Area Central Area Eastern Area


Central Area Network

To Eastern Area

To Western Area

Digital Traffic Network

• Modified hybrid mesh network built on PACTOR methods. International in scope (connections to Europe, Asia, Oceana, etc.).

• Operational 24-hours per day.

• All RF – no Internet.

• Interoperable with voice, CW circuits, and local digital networks via the Digital Traffic Station (DTS) function.

• Backwards compatible from Pactor 3(4) to Pactor 1.

• VHF gateways can be easily established at county/municipal level



“Area Backbone”

“Region Hubs”

“Local Access Points”

“State Hubs”

DTN is modern and easily implemented


Photo courtesy J. DeAngeloPhoto courtesy Timothy Eldridge

Radiotelegraph & Radiotelephone

• Universal common-denominators.

• Forward-deployable as man-pack, portable, mobile, etc.

• CW well suited to low-power and renewable energy resources.

• Voice-capable transceivers are ubiquitous.

• Efficient in the hands of qualified operators.

• Basic HF modes work everywhere.


Photo courtesy Portable Zero, LLC

Central Region Network Topography


Inter-Area Traffic Network Topography


Point-to-Point CircuitsArea Networks

Prototype National Response Plan

• Special IATN routings to FEMA NRCC via watch (QSX) frequencies spaced between 3 and 30 MHz.

• QSX frequencies assigned for exercise priority and exercise emergency traffic only.

• IATN ran parallel to automated Digital Traffic Network.– Common denominator access.– Order wire service.– Overflow message traffic.– Redundancy.


Network Topology

• Multiple injection points speed priority or emergency traffic – spread traffic load.

• Network failures easily and seamlessly bypassed.

• Full interoperability between modes and network layers.

• Interoperable features are scalable to dynamically respond to varying traffic loads and priorities.


Communications Security• Frequency and network management plan distributed only

to operational units and individual participants.

• Frequency/Mode Matrix not published on the Web.

• Considerable research was required to identify suitable allocations for the prototype plan.

• Frequency designators used to identify QSX frequencies on-air.

• Radiotelegraph and digital networks resistant to public intercept.



Sample Portion of State/Region Net Frequency Matrix

DTN Partial Mode/Frequency Matrix


IATN Mode/Frequency Matrix



HF phone / cw

HF Pactor

Internet HTTPS

BackgroundDoes the emergency plan work?

Cascadia Rising Exercise Design Philosophy

Designing a Communications Exercise

Important questions must be answered:

1. Is connectivity established and maintained?

2. Is the network survivable?

3. Is the network capable of accurately conveying information?

4. Is the information conveyed in a timely manner?

5. Is the proper accountability and network management data in place?

Most emergency management exercises answer ONLY the first question, but we need to examine the entire communications “chain.”


The Emergency Communications Pyramid


An Analogy – Train Control Network


Train Detection SIL4

Interlocking Processor

SIL4

Control RelaysSIL1

Wayside SignalsSIL4

Total Safety Level =

SIL1

Train Control Network + Human Factor


SIL4 Train Control Network

The highest level of technological safety can becompromised by human failure. The same is true of theemergency services communications process.

A lack of training in effective communications procedures undermines any technological solution.

Human Factors vs. Technological Factors

• Most communications specialists concentrate on technological solutions.

• Technology alone does not solve the “human interface” problem.– Data entry errors; “garbage in – garbage out.”– Misunderstandings related to technical terms.– Loss of accountability data (authority, position, etc).– Temporal misplacement or date-time group errors.– Inability to catalog and reference message traffic from a file.– Inability to keep a running radio log with understandable summaries– Lost messages.

Never seek a technological solution until managerial problems are resolved.


What size Aspirin?


8115.615.6 g 81 mg

One missing letter can make a difference!

What are we testing?• The exercise must test the entire communications process

including both the technology and the methodology of communications. The latter contains a significant human element.

• Technology = Medium whereas Communications = Entire Process.

– The exercise must be realistic.– The exercise must be measurable.– The goals must be attainable.

The timely delivery of accurate messages, whether by cellular data network or carrier pigeon is what matters!



Medium vs. Final Product

Above: Press telegraphers at the infamous Lonergan murder trial in New York City – 1943

Left: Carrier pigeons used to carry message traffic during World War Two

Cascadia RisingExercise Design & Evaluation


Collecting Data - Metrics• Controlled inject messages were developed to determine accuracy

through comparison to the delivered product.

• The time at which each message was injected was controlled and accurately logged.

• The time at which messages were relayed or transferred between networks, and to/from whom were logged at various points throughout the network.

• Network topology could then be defined by tracking message flow. This allows evaluators to identify network (or human) failure points.

A message may be transferred between several networks to achieve “last mile” connectivity or to bypass network failures.


Cascadia Rising Message Origination• Preformatted messages and guidance documents distributed to operators

in Alaska, Idaho, Northern California, Oregon and Washington State.

• Messages properly formatted in standard “radiogram” (“ICS213 on steroids”) format.– Accountability– Essential network management data– Temporal context (date-time group)

• All message text consisted of five-letter cipher groups.

• Occasional minor variations introduced into a small quantity of messages to test “assumptions.” Example: inclusion of a six letter cipher group or inclusion of a counter-intuitive date-time group or group-count (check).

• Original messages contained in sealed, date-time stamped envelopes indicating time to be presented for origination (injection into exercise).



Typical exercise packet distributed in Field

Message Logs and Exercise Guidelines Documents

Sealed messages to be opened & origination at

time indicated

SASE for returning all message copies and logs at

exercise conclusion

Message Destination

• Requirement: Route was entirely RF from Pacific Northwest to final delivery to FEMA National Response Coordinating Center.

• Messages delivered only upon reaching the destination area. Grid-down scenario.

• WebEOC used to facilitate message delivery and to provide accurate, written and date-time stamped record of final product as delivered upon network exit.

• Carefully vetted operators served as liaison to NRCC to limit skill variables to assets in the field.


Network Management Messages

• Large scale disasters require knowledge of network topology and connectivity:

– On what comm. network is a specific agency/official represented?

– What frequencies, modes/methods are available within a particular communications unit?

– What unique operational limitations does an individual operator/agency have (limited fuel supply, staffing limitations, frequency limitations, transmitter failures, etc.).


Network Manager• Exercise participants were required to originate periodic status

messages at specific times to an assigned network manager.

• Network manager maintained a database of operator capabilities and to generate a picture of network topology for reply message routing, access to served agencies and specific officials.

• Of particular interest were alternate routings (paths) and last-mile capabilities.

• Network manager was located off-site (not at NRCC).

• Because of their dynamic nature, network management messages were reviewed from a qualitative standpoint, but NOT scored (i.e. controlled injects unavailable).


Measurements

• Text of messages as delivered were compared against control copies of the messages presented for origination and injected within the simulated disaster area.

• Messages scored according to specific, objective criteria.

• Both quantity of errors and error pattern determined fatal or nonfatal errors.

• Time from origination to time of delivery recorded for each message.

• Connectivity failures, delays, significant outliers noted.


Grading Criteria

Cipher groups require analytical grading scheme:

1. Each letter or figure within the radiogram was treated as a unique data point.

2. Each distinct error, such as an improperly transcribed letter or figure was graded as a single error.

3. Up to three errors associated with different cipher groups were treated as individual errors. Four or more errors within an entire message were treated as a fatal error and the message was discarded.

4. Two errors within a single group (five letter cipher group or word) were counted as five errors.

5. Three errors within a single group were counted as a fatal error and the message was discarded.


Sample MessageOriginated vs. Delivered


Controlled Inject Message

Message as delivered

Typical completed log


Analysis

How did the system perform?

Some surprising results were obtained and important lessons learned.

Accuracy Measurements

Alaska Intrastate PACTOR Network: 100 %

Digital Traffic Network: 99.997%

Combined Radiotelegraph: 99.998%

Radiotelephone: Voice methods not used beyond state level and were therefore not measured.


Message Propagation TimesRadiotelegraph

Measured from time message was originated in disaster area until the time it appeared in the data stream at NRCC:

– Average message propagation time: 30.5 minutes.

– A few statistical outliers due to human error. For example; one misplaced message resulted in delay of 110 minutes.

– Remove several outliers, and average total message propagation time drops into 10 to 15 minute range.

– One message propagation time measured at 1 minute!


Message Propagation TimesAlaska Intrastate Network

• Hybrid Network. Intrastate traffic transmitted using PACTOR to a radiotelegraph gateway at Fairbanks, AK.

• Radiotelegraph circuit from Alaska to Region 7 Washington State (appx. 2000 mile circuit).

• Traffic then routed via IATN to FEMA NRCC.

• Alaska message propagation times integrated into IATN (radiotelegraph) network statistics.


Message Propagation TimesDigital Traffic Network

• Average total message propagation time: 82.4 min.

• Marginal radio frequency conditions during exercise phase one and three resulted in connect failures.

• Co-channel and adjacent channel interference: Heavy demand on the automated sub-bands resulted in excessively long delays due to carrier detect holds. DTN shares small slice of spectrum with WL2K.

• WL2K frequencies used at local/state level – “all eggs in one basket.”

• No manual attendants (sysops) present at DTN hubs (by exercise design).


Environmental Influences:Poor RF conditions negatively impacted DTN performance


RF Conditions – CONUS

A Sample of Metrics


Lessons Learned


Lessons from exercise results…

• Radio operator skill and experience is the key to success.

• A diversity of modes is essential to an effective disaster communications strategy.

• Manual modes (radiotelegraph) proved more effective due to the ability of operators to respond dynamically (and instantaneously) to frequency occupancy and poor radio frequency conditions.

• Common-denominator methods offer versatility and universal access.


Lessons continued….

• Digital Traffic Network (DTN) is less agile. It therefore requires additional hubs with better geographic dispersal to accommodate poor radio frequency conditions.

• During major disasters, an attendant (sysop) should be present at hubs to monitor performance and message throughput.

• Higher power capabilities and, if possible, directional antennas to target a disaster zone would be beneficial at region and area hub facilities.

• By staffing hubs, alternate frequencies outside the automated sub-bands could be employed in time of emergency. NECRP to be modified accordingly.


Lessons continued…

• Ultimately, the human element proved most important:

– Experience played an important role. Participant backgrounds included US Army Security Agency, US Navy, Maritime Radio Officers, and radio amateurs with extensive, daily experience in message handling.

– It is easy to misplace a message when large quantities are manually transferred between networks. Good administrative skills are essential to success.

– Multi-operator facilities outside the affected area would be beneficial.

– Greater depth of operators would be required to support an actual event over a multi-day/24-hour operation….more volunteers.


Lessons continued

• Additional network management data associated with the radiogram format afforded improved tracking and administration.

• Survivable high frequency radio networks hold significant promise for basic emergency communications response and situational awareness reporting.

• Ultimately, the most survivable networks proved most effective. In order of importance:– Survivability– Flexibility– Circuit Capacity– Security


Contact Information

James WadesRadio Relay InternationalPO Box 192Buchanan, MI. [email protected]@gmail.com

All contents copyright 2017 Radio Relay International – All rights reserved.


mailto:[email protected]

mailto:[email protected]