1 hurricane recovery experience john quigley director network integrity network operations november...
TRANSCRIPT
1
Hurricane Recovery Experience
John QuigleyDirector
Network IntegrityNetwork Operations
November 18, 2004
2
Who was the first pilot to intentionally fly into a hurricane's eye?
A. Chuck Yeager
B. Charles Lindbergh
C. Amelia Earhart
D. Joseph Duckworth
Hurricane Trivia
3
Overview
• Series of four hurricanes hit the state of Florida between August 13 and September 30, 2004– Charley– Frances– Ivan– Jeanne
• The loss of resources (e.g. backhaul, power, fuel) to individual cell-sites occurred in the Southeastern markets and hampered wireless service restoration efforts
4
Industry Issues
1. Power to remote areas slower to be restored due to geographic location Priority to restoration is unclear
2. Storm caused shortages of fuels needed for generators No priority for NS/EP entities
3. Pumps at fuel stations in the field were electric and as such would not work during major power outages following the storm.
5
Industry Issues
4. Evacuation in affected areas causes longer outage duration.
5. T1’s went offline following loss of power to central offices causing need to reprioritize work continually.
6. Circuits went down due to lack of back-up power at remote terminals (e.g. remote fiber-mux)
6
Lessons Learned:
• Backup power is critical and the most vulnerable component of telecommunications network
• Expertise in the area of logistics cannot be underestimated!– Moving people, generators, trucks– Food, Fuel, lodging
• Continual learning during and after each event using a well defined event analysis process is critical to the maturity of a BCP program
• The basis for a successful response effort starts with your ICS structure and ability to communicate at all levels during a disaster
7
Sprint’s ICS/EMG Structure
Senior Vice PresidentNetwork Serv ices
VP Field Operations VP Network Operations
Emergency ManagementGroup Director
(John Quigley)
Logistics Section
Incident Commander(Field Operations Director)
Inf ormation Of f icer
Saf ety Of f icer
Liaison Of f icer
Circuit RestorationSection
Planning SectionImmediate Response & Restoration
Response Manager(Area Manager)
TransportEngineering
Data NetworkEngineering
Voice NetworkEngineering
CapacityPlanning
NetworkInstallation
NumberAdministration
Wireless Sites
AccessManagement
Access Serv iceDeliv ery
EMMC/SMC Prov isioning Translations
CorporateSecurity
Finance Human
Resources Legal Fleet
Administration Enterprise
PropertyServ ices
Supply ChainManagement
Onsite Commander(First Level Manager, or most
senior Sprint employee atlocation)
Response Team-Sw itch/RF/Fiber/CPE Technicians-InstallationTechnicians-RF Engineering
-Public Utilities
Offsite StagingArea
(if necessary)
Note: Field OpsPersonnel areassigned to thesepositions based onevent.
CommunicationsPersonOne person assignedto stay on conference
bridge at all times.
Unif ied Command
Customer Relations Of f icer
Note: The NetworkControl Centersprovide informationand support to theEMG LiaisonOfficers and EMGCircuit RestorationSection. NCC WNMC TSA ASA GSA BMC Packet Ops SS7-INS
NTAC National
Platf orms Facilities
Engineering &Mgmt
OSS RPMNote: These groupsare typicallyengaged prior to adisaster beingcalled.
8
Actions to Consider
How can the Telecommunications industry and power industry best work together to improve reliability?
What level of protection should the Telecommunication Industry assume for remote sites?