at&t developer program · · 2012-02-032.4.1.2 blackberry pin-to-pin messaging ... and...

© 2007 AT&T Knowledge Ventures

AT&T Developer Program

Business Continuity: Disaster Preparedness and Recovery

White Paper

Legal Disclaimer

© 2007 AT&T Knowledge VenturesIntellectual Property

All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.

ii

This document and the information contained herein (collectively, the "Information") is provided to you (both the individual receiving this document and any legal entity on behalf of which such individual is acting) ("You" and "Your") by AT&T, on behalf of itself and its affiliates ("AT&T") for informational purposes only. AT&T is providing the Information to You because AT&T believes the Information may be useful to You. The Information is provided to You solely on the basis that You will be responsible for making Your own assessments of the Information and are advised to verify all representations, statements and information before using or relying upon any of the Information. Although AT&T has exercised reasonable care in providing the Information to You, AT&T does not warrant the accuracy of the Information and is not responsible for any damages arising from Your use of or reliance upon the Information. You further understand and agree that AT&T in no way represents, and You in no way rely on a belief, that AT&T is providing the Information in accordance with any standard or service (routine, customary or otherwise) related to the consulting, services, hardware or software industries. AT&T DOES NOT WARRANT THAT THE INFORMATION IS ERROR-FREE. AT&T IS PROVIDING THE INFORMATION TO YOU "AS IS" AND "WITH ALL FAULTS." AT&T DOES NOT WARRANT, BY VIRTUE OF THIS DOCUMENT, OR BY ANY COURSE OF PERFORMANCE, COURSE OF DEALING, USAGE OF TRADE OR ANY COLLATERAL DOCUMENT HEREUNDER OR OTHERWISE, AND HEREBY EXPRESSLY DISCLAIMS, ANY REPRESENTATION OR WARRANTY OF ANY KIND WITH RESPECT TO THE INFORMATION, INCLUDING, WITHOUT LIMITATION, ANY REPRESENTATION OR WARRANTY OF DESIGN, PERFORMANCE, MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, OR ANY REPRESENTATION OR WARRANTY THAT THE INFORMATION IS APPLICABLE TO OR INTEROPERABLE WITH ANY SYSTEM, DATA, HARDWARE OR SOFTWARE OF ANY KIND. AT&T DISCLAIMS AND IN NO EVENT SHALL BE LIABLE FOR ANY LOSSES OR DAMAGES OF ANY KIND, WHETHER DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, PUNITIVE, SPECIAL OR EXEMPLARY, INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF BUSINESS PROFITS, BUSINESS INTERRUPTION, LOSS OF BUSINESS INFORMATION, LOSS OF GOODWILL, COVER, TORTIOUS CONDUCT OR OTHER PECUNIARY LOSS, ARISING OUT OF OR IN ANY WAY RELATED TO THE PROVISION, NON-PROVISION, USE OR NON-USE OF THE INFORMATION, EVEN IF AT&T HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH LOSSES OR DAMAGES.

Revision History



iii

© 2007 AT&T Intellectual Property. All rights reserved.

AT&T and AT&T logos are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All marks, trademarks, and product names used in this document are the property of their respective owners.

Table of Contents



iv

1. Introduction .............................................................................................................................................1 1.1 Audience.......................................................................................................................................2 1.2 Contact Information ......................................................................................................................2 1.3 AT&T Resources ..........................................................................................................................2 1.4 Abbreviations and Acronyms........................................................................................................3

2. Preparing for Disaster and Planning for Business Continuity ................................................................4 2.1 Importance of Communications in a Crisis ...................................................................................4 2.2 The Types of Disasters Customers Should Prepare For..............................................................4

2.2.1 Physical and Natural Disasters........................................................................................5 2.2.2 Pandemics .......................................................................................................................5 2.2.3 Economic Upheavals and Labor Strikes..........................................................................5 2.2.4 Political Instability and Terrorism.....................................................................................6 2.2.5 Security Breaches............................................................................................................6

2.3 Maintaining Preparedness............................................................................................................7 2.3.1 Identification of Critical Business Processes ...................................................................7

2.4 Solutions That Reduce the Risk of Downtime Through Mitigation ...............................................8 2.4.1 Mobile Availability ............................................................................................................8

2.4.1.1 The AT&T Crisis Phone Program....................................................................9 2.4.1.2 BlackBerry PIN-to-PIN Messaging ................................................................10 2.4.1.3 Enterprise Paging..........................................................................................10

2.4.2 Redundancy...................................................................................................................11 2.4.2.1 Critical Messaging Infrastructure...................................................................11 2.4.2.2 AT&T StorageConnect Service .....................................................................11 2.4.2.3 AT&T Remote Vault Service .........................................................................12 2.4.2.4 AT&T Hosting Services .................................................................................12 2.4.2.5 LaptopConnect ..............................................................................................12 2.4.2.6 WWAN Connect ............................................................................................13

2.4.2.6.1 Using WWAN Connect for Backup Communications 13 2.4.2.6.2 Using WWAN Connect as a Portable Hotspot 13 2.4.2.6.3 Using WWAN Connect as a Mobile Command Center 14

2.4.2.7 Enterprise Recovery Services.......................................................................15 2.4.3 Security and Resilience .................................................................................................16

2.4.3.1 Commercial Connectivity Services................................................................16 2.4.3.2 Device and Firewall Protection......................................................................17

2.5 Risk Management Assessment ..................................................................................................18

Table of Contents



v

2.5.1 AT&T Business Continuity Professional Services .........................................................18

3. Wireless: An Important Element of Disaster Preparedness and Recovery..........................................20 3.1 Wireless Technology ..................................................................................................................20 3.2 Survivability.................................................................................................................................20 3.3 Flexibility .....................................................................................................................................21 3.4 Faster Recovery .........................................................................................................................21

4. AT&T Corporate Business Continuity Planning....................................................................................22 4.1 AT&T Business Continuity Efforts and Preparedness................................................................22

4.1.1 Corporate Business Continuity Planning .......................................................................22 4.1.2 Hurricane Preparedness................................................................................................23 4.1.3 Pandemic Preparedness ...............................................................................................23

4.1.3.1 Planning Now Is Essential.............................................................................24 4.1.3.2 Review and Expand Existing Business Continuity Plans to Include

Landscape of Threats ...................................................................................24 4.1.3.3 Rely on Credible Sources .............................................................................24 4.1.3.4 Develop Planning Phases with Trigger Points ..............................................24 4.1.3.5 Build Escalating Scenarios and Perform Simulation Exercises to Identify

Gaps..............................................................................................................25 4.1.3.6 Communications Is Crucial ...........................................................................25

4.2 The Wireless Network—Minimizing Downtime...........................................................................25 4.2.1 Wireless Network Services ............................................................................................26 4.2.2 Network Disaster Recovery Service ..............................................................................26

4.3 Wireless Processes ....................................................................................................................27 4.3.1 Purpose..........................................................................................................................27 4.3.2 Scope.............................................................................................................................27 4.3.3 Focus .............................................................................................................................27 4.3.4 Experience.....................................................................................................................28

5. Case Studies ........................................................................................................................................30 5.1 Customer Examples ...................................................................................................................30

5.1.1 Commercial Bank ..........................................................................................................30 5.1.2 Large Retailer ................................................................................................................30 5.1.3 Insurance P&C Leadership............................................................................................30

5.2 AT&T Examples..........................................................................................................................30 5.2.1 Hurricanes Charley, Frances, and Ivan .........................................................................30

Table of Contents



vi

5.2.2 September 11, 2001 ......................................................................................................31 5.2.3 Hurricane Katrina...........................................................................................................31 5.2.4 Hurricane Wilma ............................................................................................................32 5.2.5 2007 Winter Storms .......................................................................................................32

6. Conclusion ............................................................................................................................................33

7. Appendix: Business Continuity Planning Checklist ..............................................................................34

Figures Figure 1. WWAN Connection...................................................................................................................... 13 Figure 2. Portable Hotspot Solution ............................................................................................................ 14 Figure 3. Vehicle Area Network Solution .................................................................................................... 15 Figure 4. CCS Architecture ......................................................................................................................... 17

Tables Table 1. Abbreviations and Acronyms .......................................................................................................... 3



1

1. Introduction

Disaster preparedness and recovery planning are essential components of an organization’s business continuity during an emergency. Disasters and other disruptive events can affect every facet of a business, but IT departments in particular need plans and procedures in place to maintain mission-critical systems.

Failure to prepare for emergencies can be very costly. For example, in a recent AT&T survey of CIOs and other senior IT executives at companies with more than $10 million in annual revenue, 30 percent reported that their organizations have suffered from disaster, with 9 percent of those companies estimating the costs of repairs and lost business at about $500,000 per day.

Taking a proactive approach to business continuity planning is the best way to minimize the impact of disasters on employees and customers. It also minimizes losses in customer confidence, market share, and revenue. The more an organization prepares for disruption ahead of time, the less likely it will suffer catastrophic impact from a disruptive event. In this way, disaster preparedness and recovery planning are an important part of risk management.

Most businesses recognize the need to have logistical plans in place in case of natural disasters or emergencies. These plans include both immediate actions that should be taken when disaster strikes and ongoing procedures that should be followed in the disaster’s aftermath and recovery period.

For a business continuity plan to work well, it needs enthusiastic support from senior managers. These managers should direct individual departments to identify their business requirements if an emergency occurs and relay them to the IT team. IT managers can then develop plans to support each department’s specific needs.

This white paper provides business continuity guidance in the areas of communications and telecommunications—including both wireless and wireline technologies--and suggests specific measures that can help customers improve their preparedness. This paper also discusses the AT&T Continuity Planning and Crisis Management Program in detail so that customers can decide for themselves if the program’s elements can be adapted or added to their own organization’s plans. And finally, this paper provides several case study



2

examples that illustrate how advance preparation helped mitigate the damage arising from very serious disasters.

1.1 Audience

This document is written for IT personnel and telecom managers who are looking for ways to enhance their organization’s preparedness for business continuity and looking for tools and services available to help their company recover from a disaster.

1.2 Contact Information

In its ongoing effort to improve disaster preparedness and planning, AT&T welcomes feedback and suggestions from its community of enterprise and government customers. E-mail any comments or questions about this document to [email protected]. Please include the document’s title, “Business Continuity: Disaster Preparedness and Recovery” in your e-mail.

1.3 AT&T Resources

AT&T provides resources to help managers, planners, developers, and other stakeholders find information about business services and technologies than can help our customer’s with their disaster preparedness and planning.

• For information about Business Continuity and Security, visit business.att.com/businesscontinuity.

• For information about Commercial Connectivity Services, visit wireless.att.com/businesscenter/solutions/connectivity.

• For developer resources, visit our devCentral site at developer.att.com.

• For information about Enterprise Paging, visit wireless.att.com/enterprisepaging.

mailto:[email protected]



3

1.4 Abbreviations and Acronyms

The following table defines the abbreviations and acronyms used in this document.

Table 1. Abbreviations and Acronyms

Term or Acronym Definition

BES Blackberry Enterprise Server BIA Business Impact Analysis BMG Business Markets Group BSC Base Station Controller CCS Commercial Connectivity Services COW Cell on Wheels CPCM Continuity Planning and Crisis Management CTIA Cellular Telecommunications and Internet AssociationEOC Emergency Operations Center ERS Enterprise Recovery Service HSDPA High-Speed Downlink Packet Access MNOC Mobility Network Operations Center MSC Mobile Switching Center NEOC National Emergency Operations Center NSDNOC National Service Delivery Network Operations Center PIN Personal Identification Number RIM Research In Motion SLA Service Level Agreement SMTP Simple Mail Transfer Protocol SNPP Simple Network Paging Protocol TAP Timed Abstract Protocol VAN Vehicle Area Network VPN Virtual Private Network WCTP Wireless Communications Transfer Protocol WHO World Health Organization WLAN Wireless Local Area Network WWAN Wireless Wide Area Network



4

2. Preparing for Disaster and Planning for Business Continuity

This section describes the types of disasters that organizations should prepare for and how communications can play a vital role when dealing with emergencies. This chapter also looks at risk management and at specific solutions that can help mitigate risk and promote business continuity when disasters occur.

2.1 Importance of Communications in a Crisis

Maintaining communications is critical to coordinating a successful response to a crisis. The availability of communications is important because it allows an organization to contact employees and emergency responders, to direct the recovery effort, and to maintain business continuity.

The ability to contact employees is particularly important because it allows the organization to make sure they are accounted for and to determine if they need any assistance. It is also important to contact employees if they are needed to help coordinate recovery efforts. Depending on the nature of the disaster, landlines may be unavailable, and cellular voice and data may be the only available means of communication.

A communications failure makes coordinating a coherent response to a crisis very difficult and can prolong recovery. It affects not only the organization, but also local, state, and federal authorities who might be involved in the recovery effort. Arranging for a diversity of communications methods in advance of a disaster gives the organization the best chance of maintaining communications during and after the disaster. The methods of communications that should be included in a comprehensive disaster plan include landline, cellular, and satellite voice and data.

2.2 The Types of Disasters Customers Should Prepare For

A wide variety of disasters can threaten an organization’s operations. Natural disasters, political unrest, work stoppages, IT system or component failures, or even localized problems such as commuting delays can cause disruptions.

The next sections describe five types of threats that AT&T believes its customers should be especially concerned about. For each of them, consider how your



5

organization would maintain communications so that managers and designated emergency personnel could contact each other, and decide which elements of your disaster plan should be put into action. (For more information about specific strategies to maintain communications, see “Solutions That Reduce the Risk of Downtime Through Mitigation.”

2.2.1 Physical and Natural Disasters

Physical and natural threats include hurricanes, tropical storms, floods, fires, earthquakes, tornadoes, thunderstorms, snowstorms, and tsunamis. These disasters can devastate cities and bring business operations to a halt. Enterprises have to prepare for these events in advance because they can occur at any time with little or no warning. These disasters also illustrate why risk assessment and business continuity planning are so essential. (For more information, see “Risk Management Assessment.”)

2.2.2 Pandemics

The threat of a global pandemic poses special concern because the potential for a disease to spread rapidly coupled with a shortage or absence of effective medications could have widespread human and economic impact. For example, the World Bank estimates that a global outbreak of avian flu could result in a $2 trillion loss–3 percent of the world’s gross economic product.

Results of a recent informal poll by AT&T of IT professionals suggest that a pandemic is exactly the kind of event that companies are not currently prepared to deal with. For instance, large numbers of employees might be unavailable for work, making it difficult for enterprises to conduct business. AT&T recommends that businesses need to conduct a risk assessment and have a plan in place to deal with a pandemic.

2.2.3 Economic Upheavals and Labor Strikes

In the event of a strike or other economic crisis, businesses need to keep labor and monetary resources in reserve. For example, managers may need to be trained in how to perform tasks normally performed by organized labor.

Additionally, if the organization’s offices and equipment are threatened by a violent upheaval, critical IT systems should be thoroughly protected, redundant, or perhaps even located in a separate facility. Choosing the last option may not



6

be as difficult as it seems: Because most organizations already have testing and development environments that support applications and software, they can easily size and scale these environments so that the environments can function as recovery sites if the primary IT systems are unavailable.

Also, servers, routers, and other critical IT equipment should have protected network connections. The organization may want to consider maintain WLANs and WWANs in case wireline systems are breached.

2.2.4 Political Instability and Terrorism

The threat of political unrest is very real in many countries. Rioting and terrorism are serious risks in certain geographic regions. In countries where the likelihood of political and terrorist threats is high, enterprises should focus on determining to what extent they should deploy critical infrastructures in those places. Companies may decide to maintain only administrative offices in these areas and place IT systems in other countries. Risk assessments should be used to help identify these issues.

Additionally, enterprises in politically unstable areas may want to furnish their employees with cell phones and other wireless devices if travel to and from the office would be dangerous.

2.2.5 Security Breaches

An increasingly common threat to organizations are security breaches, which can come from either external or internal sources and which affect either physical or information assets. In terms of preparedness and mitigation, organizations should make sure that they grant employees access only to areas where the employees need to be. Similarly, workers should have access only to the applications and information that they need for their specific jobs.

When breaches are discovered, organizations need procedures in place to immediately notify security and isolate any affected systems.

In terms of application development, organizations also need to design appropriate security features into their applications and enforce security policies on their computers and wireless devices. For more information, visit devCentral and click the Security link.



7

2.3 Maintaining Preparedness

All companies, regardless of size, need to prepare for both disasters and the diminished business activity that often results from them. To be prepared, organizations need to fully identify (or inventory) their critical business components and effectively manage the risk that these components are subject to. These activities should not be a one-time event but an ongoing process. Companies need to periodically reevaluate their disaster plans, inventory new at-risk systems, and formulate new response procedures to make sure they are fully prepared for the next disaster.

Taking an ongoing and proactive approach to business continuity is essential for preparedness. Plans should specify redundant systems, backup sites, communication methods, and alternative work sites. They also should include a process for maintaining customer communications.

2.3.1 Identification of Critical Business Processes

Identifying critical business processes and how they might be affected by a crisis is a very important part of disaster planning and preparedness. This involves understanding what functions are critical to the organization and how different disaster scenarios could disrupt business continuity.

The following steps can help your organization identify its critical business processes:

• Business Impact Analysis. Before developing a disaster recovery and business continuity plan, customers should perform a Business Impact Analysis (BIA). The BIA will identify what the costs will be in terms of lost revenue, the effects on employees and customers, and even the public’s perception of your business brand. Using a BIA, customers can also estimate the costs of rebuilding and restoring service.

• Risk Assessment. Using statistical analysis and risk assessment, organizations can predict the likelihood of their data center or applications being adversely affected by a disruptive event. Such an assessment can also help IT staff plan and deploy an effective data backup and recovery solution.

• Contingency Planning. Contingency plans should be developed for the organization itself to ensure continuity of critical business operations as



8

well as for how the organization will work and communicate with key suppliers and vendors. Depending on the organization’s size, separate contingency plans may be required for different business units, and every business unit manager should be responsible for making sure that critical business functions are supported.

• Testing and Certifying. Business continuity plans must be tested regularly and done in a way that’s as “real world” as possible to ensure that they will be effective when disaster strikes. This requires developing a test plan, which involves not only conducting table-top simulation exercises but also actual recovery procedures to make sure they’ll work effectively in a crisis.

• Monitoring and Improving Performance. Enterprises should consider how changes to the business environment could affect their preparedness. To ensure that a plan will work when disaster strikes, managers should treat business continuity as an organizational priority and require that departments review the plans regularly.

2.4 Solutions That Reduce the Risk of Downtime through Mitigation

Disasters cannot be avoided, but organizations can take steps to mitigate the damage and reduce risk when they occur. This section describes specific solutions that will help keep critical systems up and running and secure, help employees and other responders communicate with each other, and help businesses resume operations as quickly as possible.

2.4.1 Mobile Availability

It is good practice to stockpile wireless devices that can be activated as needed during a disaster and its aftermath. It is also a good idea to provide these devices to parts of the enterprise that don’t normally use wireless devices. During a disaster, managers need to be able to communicate with all departments. And of course, anyone who might have to participate in disaster response must have a mobile device..

Another technology worth considering in enterprise disaster preparedness is the BlackBerry® handheld device by Research In Motion (RIM), or similar mobile devices. BlackBerry devices each have a unique PIN, enabling device-to-device (called PIN-to-PIN) messaging, for example, which provides communications



9

capability between devices even if the data center is out of service and the BlackBerry Enterprise Server (BES) is unavailable. For more information, see “BlackBerry PIN-to-PIN Messaging” below.)

Additionally, companies should consider implementing Enterprise Paging in case both e-mail and voice communications are unavailable. (For more information, see “Enterprise Paging” below.).

2.4.1.1 The AT&T Crisis Phone Program

Damaged or lost phones are one of the biggest challenges facing businesses in the wake of a disaster. To make sure that phones get into the hands of first responders and government agencies, AT&T developed the Crisis Phone Program. The program provides customers with devices that remain in “suspended mode” but can be activated when a crisis occurs.

This is how the AT&T Crisis Phone Program works:

• Devices are suspended (called “voluntary suspend mode”) for a predetermined number of days with a recurring charge of one cent per month.

• If a device needs to be activated, customers contact an AT&T representative who immediately submits a request for activation.

• When a device is activated, calls are billed at the contracted monthly and usage rates until the device is re-suspended.

• As part of the voluntary suspend process, devices are provisioned and shipped in suspend mode.

• Devices in the program’s voluntary suspend mode must be bought at non-commit pricing (the price they would cost if they were purchased without a contract).

• If an enterprise prefers to use contract pricing, calls on the devices can be billed only at the lowest available published rate.

Because enterprises have the option of keeping the devices in their possession only for emergency use, they avoid the burden of having to place an order after a disaster occurs. Similarly, they don’t have to worry about device delivery and distribution.



10

2.4.1.2 BlackBerry PIN-to-PIN Messaging

Every RIM BlackBerry device is assigned a unique eight-digit PIN. With PIN-to-PIN messaging, BlackBerry users communicate directly with each other when e-mail networks are down or voice networks are clogged. Because PIN messages are not routed through an e-mail server, BlackBerry users can still send and receive text messages in the event of e-mail network failure. This makes the BlackBerry an ideal device for emergency situations in which all communication network services are down.

2.4.1.3 Enterprise Paging

E-mail and voice communications may be cut off during a disaster, especially if the e-mail servers are located in the disaster zone. AT&T Enterprise Paging provides another communications channel when e-mail and voice communications are unavailable.

Because Enterprise Paging uses the control channel of the cellular network, it may work even when a data or voice connection cannot be established. To further enhance performance, Enterprise Paging uses dedicated messaging gateways, and it supports all standard paging protocols, including SNPP, WCTP, TAP, and SMTP. This means that consumer gateway bottlenecks are avoided—high-traffic volumes are inevitable on consumer gateways during a crisis situation.

Enterprise Paging offers the following additional benefits:

• Enterprise-Grade Performance. The system is highly reliable, with 99.2 percent of messages delivered within two minutes to devices in good coverage areas.

• Easy to Use. The Enterprise Paging solution is a plug-and-play product, which works with most standard paging applications. No complicated provisioning process is required: simply change the paging operator address to AT&T Wireless.

• Easy to Afford. The solution features a simple, easy-to-understand pager-like pricing model that’s based on a flat-rate monthly recurring



11

charges with unlimited pages, with replies billed at standard text messaging rates.

• Enhanced Messaging Functionality. Enterprise Paging supports long messages (up to 456 characters), delivery receipts, and two-way messaging.

As part of a diversity of communication methods, Enterprise Paging offers a highly reliable, cost-effective way to deliver data to and from the disaster zone when other communication methods fail.

2.4.2 Redundancy

A key element of business continuity is redundancy of critical IT systems. External services can play an important role in this. The next sections describe a variety of services that can provide redundancy for messaging, storage, network communications, and other IT functions.

2.4.2.1 Critical Messaging Infrastructure

If your organization use RIM BlackBerry Enterprise Servers or Motorola Good Technology, it is important to have redundant server architectures for to support these systems’ failover requirements.

E-mail continuity is also important, as it may be among the best and most reliable forms of communication in a disaster zone. AT&T’s Enhanced Data Rates for GSM Evolution (EDGE) network architecture provides dedicated time slots for data communications, while AT&T’s High-Speed Downlink Packet Access (HSDPA) network provides dedicated channels so that if the voice network becomes overcrowded, the data network should still be available.

2.4.2.2 AT&T StorageConnect Service

For organizations that need help with networked storage and backup, AT&T offers its StorageConnect Service. StorageConnect helps companies extend remote replication, backup, and storage-area network (SAN) solutions between corporate locations or AT&T Internet Data Centers, over any distance and regardless of bandwidth or storage protocol. This service also gives companies confidence that they’ll remain in compliance with industry and government



12

regulatory standards, and the service’s performance is backed up by some of the best service level agreements (SLAs) in the business.

2.4.2.3 AT&T Remote Vault Service

Remote Vault Service lets companies back up data from servers, PCs, or laptops to a secure off-site storage facility using a broadband Internet connection. It supports hourly, daily, weekly, or on-demand automated backups; permits easy restores without IT assistance; and runs backups in the background so that system performance isn’t degraded. The result is rapid and easy data recovery from either an alternate corporate location or a disaster recovery facility.

2.4.2.4 AT&T Hosting Services

At AT&T, we can manage your critical services from one of our advanced data centers. These centers are located worldwide and have flexible connectivity options. AT&T has the ability to design, implement, monitor, and manage your company’s applications and Web hosting. With AT&T Hosting Services, your enterprise won’t have to worry about losing critical IT functionality during a disaster—all your systems and data will be available as soon as you’re able to reestablish a connection to the AT&T network.

2.4.2.5 LaptopConnect

In the event of a disaster, employees may not be able to travel to their offices and may have to work from home or from temporary accommodations. Providing employees with LaptopConnect cellular data cards and laptop computers can keep key personnel productive and connected, wherever they are.

LaptopConnect is an ideal solution for employees who are widely dispersed and unable to share network connections. LaptopConnect provides good data speeds and excellent coverage.

The AT&T BroadbandConnect HSDPA network offers broadband speed in several metropolitan areas. Plus, AT&T’s commitment to quick recovery from disasters means that connectivity to our network may be possible before other networks are up and running.



13

2.4.2.6 WWAN Connect

WWAN Connect provides data connectivity backup and communication path diversity for remote locations, such as satellite offices or retail locations. Implementing WWAN Connect increases the likelihood that communications will still function at the remote location even if there is serious damage to the terrestrial network.

2.4.2.6.1 Using WWAN Connect for Backup Communications

WWAN Connect can to support point-of-sale backup, for example, small-office backup, ATM cash machine backup, and portable ATMs.

Figure 1 shows how a typical WWAN Connect backup solution is set up.

Figure 1. WWAN Connection

2.4.2.6.2 Using WWAN Connect as a Portable Hotspot

WWAN Connect devices can serve as a portable hotspot, such as a temporary office location, to keep employees productive during and after a disaster. Getting a terrestrial data connection into a temporary office during a disaster will be difficult if not impossible.



14

The portable hotspot provides instant connectivity. Setup is simple: just plug in a wireless access device and position it for the best cellular reception. WWAN routers have Wi-Fi and Ethernet ports for local connectivity, providing considerable flexibility of deployment. This configuration differs only slightly from the backup solution.

Figure 2 shows how WWAN Connect can be set up as a portable hotspot.

Figure 2. Portable Hotspot Solution

2.4.2.6.3 Using WWAN Connect as a Mobile Command Center

WWAN Connect devices can also be used to equip mobile command centers or other vehicles to provide a hotspot on wheels for use in the disaster zone. The device acts as a WLAN-to-WWAN bridge, providing local Wi-Fi service and backhauling it over the cellular network. WWAN router devices are already in use at many public safety and law enforcement agencies. The mobile command center is also known as a vehicle area network (VAN).

Figure 3 shows how WWAN Connect can be set up as a VAN.



15

Figure 3. Vehicle Area Network Solution

2.4.2.7 Enterprise Recovery Services

AT&T Enterprise Recovery Service (ERS) offers a full portfolio of subscription-based disaster recovery services for systems and user work locations to help companies deal with any unplanned event that threatens their profitability.

ERS takes advantage of AT&T's expertise in networking, computing, data mirroring, and IT infrastructures. It is built on a well-established presence that includes metro-area networking availability and data-center and recovery facilities in major markets.

ERS ranges from recovery of distributed and legacy systems, to provisioning of alternate workspaces. It can even include rapid deployment of mobile recovery units that support multiple platforms and communications applications. AT&T ERS offers comprehensive cost-effective alternatives to in-house IT approaches.

AT&T works in partnership with SunGard Availability Services to provide customers with access to critical data and systems. ERS include the following core components:

• Traditional Recovery. When AT&T finds a customer that has a business continuity requirement, it works with a SunGard channel manager to develop a solution. SunGard offers traditional recovery services through facilities located throughout the United States.



16

• Managed Services. When AT&T cannot offer physical data center space, it provides managed services. In these instances, the company uses SunGard floor space for AT&T customers. Services include security, replication, and storage.

• Professional Services. AT&T works with a SunGard professional services team to provide risk assessment and test services.

2.4.3 Security and Resilience

As enterprises create plans for disaster preparedness, they want the confidence of knowing they’ll have secure and resilient communications during a crisis.

AT&T provides a variety of solutions designed to meet these enterprise needs, in both day-to-day business and in emergency situations. Two of the most important of these solutions are Commercial Connectivity Services (CCS), and device and firewall protection offered by AT&T Security Services. The next sections describe these solutions in detail.

2.4.3.1 Commercial Connectivity Services

AT&T Commercial Connectivity Services is a solution that allows a company to capitalize on its existing infrastructure as new functionality is added that enhances security.

In the context of disaster preparedness and secure, reliable communications, a company can choose to implement a frame relay connection or a virtual private network (VPN). Both solutions reduce the risk of slowdowns or service interruptions that may result from especially heavy data traffic volumes that occur during a disaster.

CCS can also provide redundancy with geographic diversity by connecting multiple enterprise data centers to multiple AT&T data centers and can reroute data if any of the data centers go offline during the disaster.

Figure 4 shows the CCS architecture.



17

Figure 4. CCS Architecture

2.4.3.2 Device and Firewall Protection

AT&T Security Services provides device and firewall network security functions that are essential for any company implementing Internet-based networking. The following functions are particularly important:

• Device. AT&T helps customers deploy endpoint security solutions for laptops, PCs, and mobile devices. These security solutions are critical to maintaining the integrity of the IT infrastructure on a day-to-day basis. Additionally, AT&T’s solutions provide customers with the ability to administer policies and compliance across all devices. This can be accomplished through the AT&T Device Protection and Control offer.

• Firewall. AT&T provides a network-based firewall solution that allows customers to access the Internet or other critical systems even if their data center is down. For example, during the Hurricane Katrina disaster, AT&T had several customers in New Orleans that had data centers completely submerged in floodwater. AT&T was able to bring those customers’ systems up on its network-based firewall within 24 hours, allowing them to continue doing business even with the data center under water.



18

2.5 Risk Management Assessment

AT&T recommends that all customers conduct a risk management assessment to determine which critical systems are most susceptible to disruption during a disaster. After conducting the assessments, customers will know which systems in their infrastructure are truly critical and which are merely important.

For critical systems, customers need to have a business continuity plan in place to mitigate the business issues that arise when those systems go offline; for other systems, customers need security and other protective measures to minimize service disruptions.

2.5.1 AT&T Business Continuity Professional Services

It can be useful to engage external resources to help with risk management assessments. AT&T has a range of best practices to help enterprise and government customers improve disaster recovery preparedness. In addition, AT&T helps customers record and identify the requirements for a business process or element (such as a billing, or the deployment of critical applications). This includes understanding hardware specifications (servers, storage equipment, disk drives, etc.); what the service-level or performance needs are for individual components; and capacity.

With the comprehensive suite of Business Continuity Professional Services from AT&T, you can make informed decisions about your business continuity programs, strategies, and investments. Whatever your industry, AT&T can analyze, design, and implement a program that helps to ensure operational readiness and the recovery capabilities necessary to keep your business up and running.

The AT&T Business Continuity Professional Services portfolio includes:

• Managed Risk Services to provide an objective appraisal of your business environment, including a Business Impact Analysis (BIA) and Risk Assessment.

• Business Continuity and Disaster Recovery Consulting to provide continuity and recovery strategies for critical business processes that focus on planning, testing, training, and certification.



19

• Business Continuity Program Management to provide program management consultation, expertise, and methodology.



20

3. Wireless: An Important Element of Disaster Preparedness and Recovery

3.1 Wireless Technology

In many disaster scenarios, wireless technology has a higher chance of surviving than landline technology. For example, when a wireline connection is anchored on both ends, it may fail if either the switching office or the termination point is in the disaster zone. By contrast, even if a particular wireless cell site or switch is down, there is still the possibility of connecting via another cell site that survived the disaster or is located outside the disaster zone.

Mobility is another obvious advantage of wireless technology. Unlike landlines that are tied to a specific location, wireless technology offers mobile data and voice access. This provides two capabilities:

• Responders can bring reliable communication technology with them into disaster zones where existing means of communication may be damaged or destroyed.

• If coverage has been knocked out in a particular location, it is possible to move to another location where coverage is still available.

3.2 Survivability

A potential problem with landlines is that they present multiple methods of failure, any one of which could interrupt communications. For example, employees needing to communicate may not have access to landline communications if they are in an area that is under water or covered in rubble. Even if they get to a landline phone, the phone switching office could be out of service as a result of the disaster.

Including wireless voice and data capabilities in a business continuity plan increases the likelihood that communications will survive in a disaster by eliminating the single-point-of-failure problem: if provisions have been made for only a single mode of communication, all communications are cut off if that one mode fails. However, if both landline and wireless capabilities are present, the chance that at least one service will remain available in a disaster increases significantly.



21

3.3 Flexibility

Wireless technology offers a degree of flexibility not offered by landlines. Because it is not tied to a specific location, it is possible to travel to areas where service is still available, to where recovery efforts are under way, or to temporary office locations, and have voice and data communications available immediately.

Also, wireless users can communicate wherever they want, from any device they want. Wireless devices support multiple forms of communication such as instant messaging, e-mail, voice mail, and data applications. Because many of these technologies use the wireless network differently, there may be times when one mode of communication is available while another is not. For example, signal strength may be too weak to make a voice or data connection, but text messaging using SMS will still work.

3.4 Faster Recovery

Restoring landline service in a disaster zone can be a difficult and lengthy process, especially if facilities and infrastructure are under water or otherwise inaccessible. Recovery efforts aimed at restoring wireless service can start and finish sooner. Two methods that can be used even before repairs to cell sites and switching centers begin are Cells on Wheels (COWs) and cell site retuning.

A COW is a complete mobile cell site located on the back of a truck that can be driven as far into the disaster zone as is viable. COWs consist of base station radios, on-board power generation, a tower structure, and antennas. They provide a way to temporarily restore parts of the network until permanent repairs can be made.

Cell site retuning involves making changes at cell sites to reduce coverage in areas that have been depopulated by the disaster, and increase coverage in the areas where people remain.



22

4. AT&T Corporate Business Continuity Planning

4.1 AT&T Business Continuity Efforts and Preparedness

AT&T has extensive experience in preparing for events such as natural disasters, which pose a major challenge because of their potential for disruption and the short response window. This experience includes tracking and reporting the impact of major storms; activating and staging resources for recovery; activating fuel cells and other equipment in preparation for early deployment; and ensuring readiness by initializing communications between all participants prior to the storm.

AT&T crisis management is handled by four regional teams: Northeast, Southeast, Central, and West. Each team creates area-appropriate disaster recovery strategies and determines which cell sites should be recovered first in the event of a crisis.

The AT&T emergency response program includes mobile command centers for satellite communications. These centers provide full network support for recovery teams. The program also includes permanent command centers and portable generators, as well as mobile retail stores that can be quickly installed to sell batteries, chargers, and other equipment.

4.1.1 Corporate Business Continuity Planning

AT&T subscribes to the Cellular Telecommunications and Internet Association (CTIA) voluntary Business Continuity and Disaster Recovery Program, and is currently in the process of becoming certified. The established program elements are:

• Project Initiation and Management

• Risk Evaluation and Control

• Business Impact Analysis

• Business Continuity Management Strategies

• Emergency Response and Operations



23

• Development and Implementation of Business Continuity Plans

• Awareness and Training Programs

• Exercise and Maintenance of Business Continuity Plans

• Public Relations and Crisis Coordination

• External Agency Coordination

As part of its planning efforts, AT&T has established a national emergency response program to coordinate response recovery efforts across all business lines.

4.1.2 Hurricane Preparedness

AT&T Wireless prepares in advance to deal with the aftermath of hurricanes. In the earliest stages of a hurricane, the network team coordinates with the company's Emergency Preparedness and Recovery Team. The storm's path is tracked by the National Hurricane Center for landfall, wind speed, and rainfall forecast. AT&T uses this data to strategically stage generators, fuel cells, and other equipment in key locations outside of the projected storm zone. The network team also coordinates with local power and landline telephone companies. Network suppliers are engaged for assistance as needed.

To support the recovery effort, AT&T delivers portable trailers as temporary housing for employees from other areas who have come to help. An Emergency Operations Center (EOC) is established at a bunkered switch location. Provisions at each EOC include food, water, first aid kits, network parts and equipment, chain saws, tarps, safety glasses, batteries, flashlights, and other essentials. The National Service Delivery Network Operations Center (NSDNOC) in Bothell, Washington, establishes a team to track and monitor every network element in the hurricane zone, 24 hours a day.

4.1.3 Pandemic Preparedness

In response to the growing concern around the risks of a pandemic influenza, AT&T launched a customer program team in December 2005 on pandemic planning. Through this program, AT&T and its customers have shared their efforts, challenges, and best practices in preparing for such a crisis.



24

According to the U.S. Department of Health and Human Services, pandemic influenza viruses have demonstrated their ability to spread worldwide within months or even weeks. In the particular case of a pandemic influenza, if an organization waits until disease mutation occurs (i.e., avian flu strain becomes easily transmissible between humans), it could be too late to react.

AT&T and its customers developed the following set of key principles regarding business continuity planning for a health threat such as a pandemic influenza. These principles can also apply to any business continuity scenario in the public or private sector.

4.1.3.1 Planning Now Is Essential

Once disaster strikes, an organization’s ability to respond quickly and effectively may be critical in protecting its staff, profits, reputation, and essential operations. Developing a plan that protects the health and safety of employees and ensures that critical business functions remain operational requires a comprehensive and cross-organizational planning effort.

4.1.3.2 Review and Expand Existing Business Continuity Plans to Include Landscape of Threats

Many organizations have business continuity plans to deal with disruptions, but they may not be prepared for an event that could occur on a global scale. Existing business continuity plans should be reviewed and supplemented accordingly to meet a range of threats.

4.1.3.3 Rely on Credible Sources

It is important to identify reliable and credible sources of pandemic information early in the planning process to track developments.

4.1.3.4 Develop Planning Phases with Trigger Points

Organizations should create clearly defined response-planning phases, with trigger points for moving from one phase to another. For example, resources such as the World Health Organization (WHO) can be used as a reference point for defining phases and trigger points for pandemic influenza planning.



25

4.1.3.5 Build Escalating Scenarios and Perform Simulation Exercises to Identify Gaps

Many companies are conducting business-continuity planning exercises using a range of scenarios to assess the impact of an avian flu pandemic on their businesses. One scenario may simulate the disease breaking out slowly; a second may assess the effects of a sudden outbreak in several locations simultaneously. Scenario-based exercises help identify gaps and risks that might not otherwise be obvious. Build scenarios starting with a small outbreak, and then move up to a worst-case scenario.

4.1.3.6 Communications Is Crucial

The ability to withstand any crisis may ultimately rest on the effectiveness of its communications with employees, clients, suppliers and other key constituents. Senior executives should be ready to deliver the right messages at the right times.

4.2 The Wireless Network—Minimizing Downtime

The two primary causes of network failure in a disaster are power outages and transmission outages. AT&T has measures in place to guard against both of these eventualities.

To protect against power failure, AT&T has backup battery power plants located at data centers, Mobile Switching Centers (MSCs), and cell sites that can provide several hours of service at peak utilization. Backup generators at the same sites provide backup power and/or recharge batteries. In worst-case situations, when significant numbers of cells are lost despite these precautions, a mobile fleet of trailer-mounted generators is deployed as needed.

Data transmission outages are outages in the terrestrial network to which cell sites, Base Station Controllers (BSC), and MSCs attach. To provide optimal redundancy and high survivability, AT&T uses a combination of T-spans, coaxial cable, fiber, and microwave links for transmission between cell sites and MSCs. The use of dedicated lines for landline links reduces the effect of heavy non-wireless traffic during emergencies. AT&T also has the ability to divert traffic from one switch or cell tower to another if necessary.

Because large-scale disasters cause significant service disruptions in the wireless network and in the terrestrial network that supports wireless, AT&T is



26

prepared to send in COWs to a location where service has been lost or requires enhancement. These resources are available to all disaster relief agencies and AT&T Wireless users in an area rather than for specific customers.

In the event that wireless and landline services are disabled, satellite phones and two-way radios are in place for communication between field technicians and incident management groups. Once the network is restored or sufficiently augmented, a pool of phones is available for emergency use. There are currently 2,500 phones allocated for the Federal Emergency Management Agency (FEMA) in case of an emergency.

4.2.1 Wireless Network Services

The AT&T Wireless Network Services team takes a number of steps to ensure maximum survivability and rapid recovery in the event of a disaster. AT&T Wireless cell sites are built to meet or exceed regulatory requirements, and are tested and proven to be robust and fault-tolerant. AT&T Network Services also maintains a comprehensive set of disaster recovery plans.

Extensive redundancy and hardware duplication are built into the Mobile Switching Centers, so if one fails, traffic can be routed to another. Equipment in the field is equipped with alarm systems to provide early warning of events that could result in loss of service. These systems include automatic alarms for fire, extreme temperature, and unauthorized access.

The networks are monitored 24 hours a day, seven days a week, by the NSDNOC and Mobility Network Operations Centers (MNOCs). MNOCs perform local monitoring and back up one another and the NSDNOC.

To ensure readiness, AT&T Network Services conducts regular simulation exercises in which engineers and technicians identify and repair simulated damage resulting from likely disasters.

4.2.2 Network Disaster Recovery Service

The AT&T Network Disaster Recovery Service includes special operations teams trained in first aid and the handling of hazardous materials. It also includes experts in providing or restoring HVAC (heating/ventilation/air conditioning) infrastructure and in setting up alternate communications networks using satellite links.



27

In the event of a disaster, the EOC is deployed and tasked with front-line coordination and staging in the affected area. It provides a single point of contact for managing and coordinating resources in the affected area. Operations directed by the EOC include field operations, switch operations, transport of technical staff and any other resources, and staging of all equipment, staff, and supplies.

To maximize the effectiveness of the EOC, AT&T has purchased a tractor-trailer mobile command center. AT&T also warehouses food, water, and other necessary supplies in anticipation of establishing a mobile command center.

In a serious disaster, there may be multiple EOCs managed by a single EOC. The task of the primary EOC is to define recovery strategy and communicate it to the other EOCs. This process includes establishing and communicating work priorities. The EOC is also responsible for coordinating with AT&T upper management and participating in working bridges as required by the recovery effort.

4.3 Wireless Processes

4.3.1 Purpose

The purpose of these AT&T processes is to identify, prevent, or mitigate risk exposures related to people, profit, processes, and property within AT&T and, by extension, AT&T’s ability to service customers. These processes are intended to maximize AT&T’s level of preparedness and to ensure the timely response and recovery of critical services across the enterprise.

4.3.2 Scope

Continuity maintenance and crisis management is an integral part of how AT&T does business and is critical to AT&T’s ability to deliver the most reliable network possible. This includes such processes as plan maintenance, exercises and drills, risk identification, proximity, and preparedness assessments.

4.3.3 Focus

The wireless program has four main thrusts: prevention, mitigation, response, and recovery.



28

• Prevention. The purpose of the prevention initiative is to identify, assess, and take action to avoid risk exposure before the risks materialize. To this end, AT&T has developed a set of standards and guidelines that are used to minimize risk. Additionally, risk and impact analyses and site risk assessments are undertaken regularly to identify the greatest risks.

• Mitigation. These activities are intended to identify ways to lessen the effects of unavoidable risk exposures to ensure recoverability. Among the mitigation activities that AT&T provides are business impact analyses, service criticality assessments, disaster recovery strategy development, and executive and strategic contingency planning.

• Response. This refers to efforts to ensure that systems, processes, and resources are ready and available to manage incidents. Response-related activities include crisis and incident management planning; disaster response support; support of crisis management centers, emergency operations centers, and disaster field offices; and coordination with governmental and other responders.

• Recovery. This encompasses activities intended to ensure preparedness for the aftermath of the disaster. Activities undertaken in support of recovery include end-to-end business recovery planning, disaster exercises, disaster readiness exercises, corrective action and business resumption plans, disaster recovery support, and coordination with local, regional, and national agencies. The objectives of these activities are to continue business operations following outages, events, and crises, and to minimize network down time.

4.3.4 Experience

AT&T has a very experienced team supporting continuity and crisis management. Representatives of AT&T’s continuity and crisis management team speak at related professional conferences and to industry organizations and have been published in the Disaster Recovery Journal. AT&T also maintains memberships in a number of disaster recovery organizations, including the Association of Contingency Planners and the Business Recovery Managers Association.

In addition, AT&T has partnered with Homeland Security, FEMA, the Office of Emergency Management, and other federal agencies to ensure that the AT&T



29

response is coordinated with other response efforts and so that AT&T can best assist in those efforts.

Additionally, the AT&T Business Markets Group (BMG) provides crisis management services to its business and governmental customers. Services available include expedited delivery of relief phones and data devices, single point of contact for emergency orders and network status updates, and a dedicated cross-functional support team that directs BMG manpower and resources during incidents.



30

5. Case Studies

5.1 Customer Examples

5.1.1 Commercial Bank

Several financial institutions were affected by the terrorist attacks on September 11, 2001. For one bank, the disaster recovery planning it did in advance enabled it to recover rapidly. Since then it has added to its support infrastructure for critical applications, ensuring that backup equipment is available to secure valuable information. It has dispersed its locations and data centers over a wider geographical area, has hired dedicated business continuity staff, and tests its systems several times a year to stay prepared.

5.1.2 Large Retailer

As hurricanes ravaged central Florida in 2005, a major department store could not process customer transactions with its landlines down and no data connectivity. AT&T flew staff into the state to hand deliver WWAN equipment to the store, providing data connectivity backup. The entire department store shared a single EDGE connection until wireline connectivity was restored.

5.1.3 Insurance P&C Leadership

In the wake of Hurricanes Charley, Frances, and Ivan, AT&T rush-ordered nearly 1,000 EDGE cards through Enterprise On-Demand to help a leading insurance carrier process the 15,000 storm-related claims that it received each day. EDGE cards from AT&T allowed adjusters to work in the field. The cards were one of the few options available to process emergency claims because the infrastructure in the region was so heavily damaged.

5.2 AT&T Examples

5.2.1 Hurricanes Charley, Frances, and Ivan

When Hurricanes Charley, Frances, and Ivan pounded the Southeast in 2004, the AT&T Network Team worked to restore service in record time. AT&T Field Technicians were able to recover more than 500 downed cell sites in fewer than



31

40 hours following Hurricane Charley, and more than 400 sites in fewer than 30 hours for Frances and Ivan.

5.2.2 September 11, 2001

Following the September 11 attacks on the World Trade Center and the Pentagon, AT&T Network Services activated the National Emergency Operations Center (NEOC) in Bothell, Washington, as well as the local EOC and Disaster Field Office in Paramus, New Jersey. Surveys were performed on each of the 46 cell sites damaged or destroyed in the attack. To compensate for lost cell sites and to accommodate the heavy call volumes, AT&T added 38 channels in Pittsburgh, 217 channels in Washington, D.C., and 1,640 channels in Manhattan. AT&T also coordinated delivery of safety kits, radios, goggles, breathing filters, gloves, and radio equipment to those working in the affected areas. Additionally, AT&T worked with the National Communications System and the United States Secret Service to provide communications access at restricted sites such as the White House and Ground Zero.

5.2.3 Hurricane Katrina

On August 28, 2005, Hurricane Katrina slammed into the Gulf Coast of the United States, badly damaging New Orleans and other cities. An AT&T Regional Crisis Response Team was ready for Katrina because it had already gone through a major training exercise that included several challenging scenarios and disasters that involved more than 300 employees in several markets around the country.

The first COWs arrived onsite within 48 hours of the disaster; eventually, 15 COWS were deployed to the area. Altogether, it took 500 generators, 800,000 gallons of fuel, 18 RVs, and almost 400 people working around the clock for approximately three weeks to repair the network. The network team restored 600 cell sites within a week and had restored most geographic coverage within two weeks.

Limited-capacity coverage was soon restored to most of the affected area. Within several weeks, the network was mostly restored but required capacity adjustments to compensate for new usage patterns (e.g., fewer people in New Orleans and more in Baton Rouge). Twenty-one days after the event, the network was back to normal.



32

5.2.4 Hurricane Wilma

Hurricane Wilma struck the Florida coast on November 24, 2005. Later the same day, when conditions were safe, more than 400 AT&T technicians, engineers, and contractors arrived in the hardest-hit areas and began installing 430 portable generators. The AT&T employees also began using COWs in the locations most affected by the storm. Within 48 hours, AT&T call centers in Miami, Ocala, and Orlando were reopened. In addition, more than 75 percent of AT&T’s South Florida stores were open for customers, although many were operating in manual mode because of widespread power outages. AT&T also set up tables outside many of its retail stores to ensure that customers could buy accessories such as extra batteries and chargers. AT&T then provided free calling stations at open stores where coverage existed. This enabled people in the disaster zone to check in with employers and get in touch with loved ones.

5.2.5 2007 Winter Storms

On January 19, 2007, a series of powerful winter storms spread freezing rain, ice, and snow across parts of the Southwest and Midwest, stranding nearly a half-million people without commercial power and thousands without phone service. In all, approximately 21,000 customers lost wireline phone service in the affected areas. Iced cell towers suffered power losses that interrupted wireless call transmissions.

In response, a small army of AT&T network managers and service technicians worked around the clock under extreme conditions to restore service to customers affected by the freezing conditions. These employees drove through the night along icy roadways and crossed snow-covered fields to deliver and connect nearly 100 generators to cell sites that would bring customers back online. Due to the dedication of the AT&T Network Services Team, wireless phone service was quickly restored in most of the affected area.



33

6. Conclusion

Businesses must develop a comprehensive plan for business continuity and disaster preparedness and recovery. When unexpected—possibly even catastrophic—events occur, you need to protect your employees and continue critical operations that support the communities that your business serves. Business Continuity Planning focuses on multiple aspects of your business, making sure you can recover the technology and processes required to operate after an unforeseen failure in normal operations.

Communications are a vital component of every business continuity plan, and there is a great deal that enterprises need to take into account to make sure that their voice and data communications remain available during and after a disaster. Wireless technology can be a key component of this strategy. With it, you can enhance the diversity of your communication paths, make your communications more resilient, provide greater flexibility to employees and recovery teams, and speed disaster recovery.



34

7. Appendix: Business Continuity Planning Checklist AT&T developed the following checklist to help in your preparedness efforts. The checklist identifies important, specific activities that businesses can do now to prepare for an event. 1. Planning for an unexpected or catastrophic event

Identify a coordinator and/or team with defined roles for preparedness and response

planning. Potential team members may come from: Information Security, Operations, Systems, Police/Security, Physical Plant, Insurance, Legal Affairs, Public Affairs, Personnel Department, Comptroller, Audit Division, and the Safety Office and/or Emergency Response Team.

Conduct a business process and services inventory to understand which processes are

mission-critical to the survivability of the business.

Determine acceptable levels of service during the recovery period, and what processes need to be maintained or restored first to keep the business running.

Identify essential employees and other critical inputs (sub-contractors, services, logistics,

etc.) required to maintain business operations by location and function during the event.

Conduct a technology asset inventory to determine and document the mission-critical technology components, their location, how they’re configured, and who is responsible for management.

Once key components are identified, determine what measures should be taken to

protect and recover them.

Understand the rules or regulations governing your business operations. If you had a business failure, would you be able to maintain compliance (Sarbanes-Oxley, HIPPA, privacy, etc.)?

Understand customer or business partner performance metrics/service level agreements

to assess risk for breach of contract, or to put in place performance remedies for your customers.

Identify a budget: Quantify the potential costs of downtime or total business failure.

Develop a business case to optimally invest in risk mitigation. 2. Assessing your data and technology needs in the event of a failure in operations



35

Determine the status of your existing disaster recovery plan. Do you have one and is it maintained? Have you tested the plan?

Determine vulnerability of your organization’s technology infrastructure to natural

disasters, including floods, fires, and earthquakes.

Set clear recovery time objectives for each of your business/technology areas.

Determine the need for off-site data storage and backup.

Develop a technology plan that includes hardware, software, facilities, and service vendors.

Secure clear understanding and commitment from vendors on your plan.

Secure a backup vendor, if necessary, to perform that critical function if your primary

vendor is affected by a business failure.

Perform security risk assessments around specific threats where possible. Examples of data security include: virus protection, intrusion detection, hacker prevention, network events, component failures, and systems crashes.

Assess, if possible and per prior events, how quickly and accurately your business and

technology were restored by existing staff. What were the lessons learned so they can be addressed in future planning?

Determine the effectiveness of your data backup and recovery policies and procedures.

Are the procedures fully documented and an appropriate staff member responsible for the maintenance of that documentation?

Perform a data recovery test. Was the test successful?

Prepare an incident plan for mitigating a security breach. Audit annually, as security

threats can change. 3. Communicating your plan to employees and vendor partners

Determine who needs to be contacted with critical information. Build distribution lists and maintain for accuracy.

Develop a contact plan to reach employees: wireless, home, etc.

Ensure that employees know where to receive information and updates about whether

they can return to work, or if they are to report to a different location (e.g., the Internet or conference bridges).



36

Ensure that mission-critical employees know their role in the plan and have access from

remote locations (e.g., through home broadband, phone, or VPN for security).

Make sure the plan can be executed by alternate employees who are not necessarily the “expert” in cases where those employees cannot be reached.

Determine the need for a designated recovery site for your people to resume work. Plan

for communications, data connectivity, desktops, and workspace at that site.

If you require support from vendor partners, ensure that they also have a documented plan that complements your needs. Review periodically to keep the plan current.

4. Coordinating with external organizations and helping your community

Collaborate with your local government agency to share your plans and understanding of their capabilities in the event of a catastrophe.

Share your plan with your building management so they have a clear understanding of

their role in safely securing the building and your employees.

Share best practices with other business leaders in your community, chambers of commerce, and business associations to improve community response efforts.

at&t developer program · · 2012-02-032.4.1.2 blackberry pin-to-pin messaging ... and...

Documents