i nnovations tomorrow’ssolutions today’s hiaa forum 2003 this presentation was given at the aahp...

IINNOVATIONSNNOVATIONS

tomorrow’stomorrow’s

SOLUTIONSSOLUTIONS

today’stoday’s

HIAA Forum 2003

This presentation was given at the AAHP / HIAA National Forum on November 11, 2003. “HIAA” – Health Insurers Association of America and “AAHP” - American Association of Health Plans members were attendees. These two associations’ members provide health care insurance to over 200 million Americans nationwide. Presentation attendees included CEO’s, line managers and operations and technology executives.

The presenters, John Schladweiler & David Adolphson, provided case study results and steps to take to ensure recoverability in the event of a disaster. If you would like to discuss this presentation, contact information is provided on page 26. Feedback is welcome.



SOLUTIONSSOLUTIONS

today’stoday’s

Planning for the Unthinkable…Disaster Recovery & Business Continuity

Las Vegas, Nevada

November 11, 2003

IINNOVATIONSNNOVATIONStoday’stoday’s tomorrow’stomorrow’s

SOLUTIONSSOLUTIONS

FORUMFORUM20032003 HIAAHIAA



SOLUTIONSSOLUTIONS

today’stoday’s

Agenda

• Introductions• John Schladweiler• Dave Adolphson

• Planning for the Unthinkable– Why Listen? – Traditional Approach– Business Issues– Best Practices – Business Impact Analysis– What Should You Do?

• Questions / Discussion

i



SOLUTIONSSOLUTIONS

today’stoday’s Planning for the Unthinkable… Why Listen?

Experience1

John Schladweiler– IT Strategy Consultant

– Former SVP of leading recovery services company

– Networked Recovery Solutions when Chicago Tunnel Flooded

– CNN Business Insight Program after AT&T Cable Cut

– WTC Bombing in 1991 – Workarea Recovery

– Top 3 Global Bank Plan for Addressing RTO

Dave Adolphson– IT Consultant

– Former VP of IT Planning for major insurance company

– Designed and Implemented Business Resumption Plans for Insurance

– Tracked 911 resumption efforts

– Directed virus disinfection and recovery efforts



SOLUTIONSSOLUTIONS

today’stoday’s

“But I already have a Disaster Recovery Plan”2

Basic ways enterprises have addressed business resumption planning without overspending

Fortress

Redundancy

Business Interruption Insurance

Planning for the Unthinkable… Why Listen?

Time

Delivery

$



SOLUTIONSSOLUTIONS

today’stoday’s


CEO Survey (April 2003):

Concerned/Not Good

Confident/Good

Source: CEO Counselors, Inc. www.ceocounselors.com

0 4321

Terrorism, Disaster

On-Time, On-Budget

Spend Right Amount

IT Management

CIO Informs CEO

IT Supports Change

IT Strategically Aligned

Competitive Advantage

Info Security/Privacy

Investment Value

3



SOLUTIONSSOLUTIONS

today’stoday’s


Old

New

Front OfficeBack Office

Real Time

Batch

Impacts:

•Payors

•Brokers

•Employers

•Providers

•Insureds

IT now supports external business processes

Changing Business Model4



SOLUTIONSSOLUTIONS

today’stoday’s

Traditional Approach

Alternate Sites5

Components:• Data Centers

-Mainframes-Servers-Infrastructure

• Network - Internal• Network - External• Offices/Call Centers

DataCenter Hot Site

Primary Office

Office 2



SOLUTIONSSOLUTIONS

today’stoday’s


Data Center– Hot Site– Cold Site– Redundant Site– Load Balanced Site– Remote Management

Network– Alternate Routes– Dial Back Up– Self Healing– Remote Management

Data Back Up– Electronic / Not Paper– Weekly, Incrementals– Real Time by Transaction– Mirrored at Alternate Site– Remote Management

Office– Workstations– Infrastructure– Call Centers– “Safe” Distance Away

Implementation Choices6



SOLUTIONSSOLUTIONS

today’stoday’s


Data Protection & Recovery Criteria7

Recovery Time Objective

How quickly must the business be back up and operational? Not all businesses are the same.

Recovery Point Objective

When data is restored for use, how current is the data? Is it to the point of the last transaction posted? Or is it to the last back up taken? Is it synchronized across computing platforms?

The Norm: Back Ups Weekly + Incrementals Daily



SOLUTIONSSOLUTIONS

today’stoday’s


Disaster Recovery Plan– Proceduralized Plan

– Multiple Contingencies / What If’s

– Tested at Least Annually

– Jointly Owned: IT & User Departments

Disaster Recovery Budget– IT Expenses Identified

– Subset of Actual Total Cost for Recovery

Recovery Plan: Tested Regularly8



SOLUTIONSSOLUTIONS

today’stoday’s

Business Issues

Why it doesn’t Work in Real Life9

Shortcomings of some existing disaster recovery plans:

Network takes too long to recover & has less capacity

Hot sites back up Mainframes, but don’t cover Servers

Timeline to re-build Server infrastructure is long

Data recovery is too slow, and requires re-entering transactions

Still dependent upon paper files

Call Centers are lacking

People

• Availability for daily operations v. recovery efforts

• Location of interruption may require staff to travel



SOLUTIONSSOLUTIONS

today’stoday’s

Business Issues

Case 1: Major Airline10

Bunker Model

Centralized business processes for Selling Tickets, Scheduling Flights & Crews

Resumption Time greater than 3 days = Bankruptcy



SOLUTIONSSOLUTIONS

today’stoday’s

Case 2: Top Money Center Bank11

Business Issues

Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo

F I I I

- F F I, I I

ThFr SSMo TuWe Th Fr Sa

BusinessProcessOutage

BackupSchedule Restore

EnvironmentProcessMissed

Days

ResumeBusinessProcess

Best Case: 12 Days!Legend:F= Full backupI = Incremental

F

Recovery Timeline



SOLUTIONSSOLUTIONS

today’stoday’s

Business Issues

Case 3: Major Insurance Company12

What would be Benefits of Improving Recovery Strategy?

from to

• Mainframe 4 days 24 hrs

• Server infrastructure - rebuild 4 days 24 hrs

(400 servers)

• Server data synchronization 14 days 5-7 days

(To 11 pm night before)



SOLUTIONSSOLUTIONS

today’stoday’s

Best Practices

Best Practices Approach13

Elements

• Redundant Working Sites• Data mirroring• Network self healing• Call Centers / Other Support• Varying RTO’s according to

business service line

Jewels

• Data - Electronic & Paper• Data - Recovery Point• Network – Data & Voice• People

- who know the business



SOLUTIONSSOLUTIONS

today’stoday’s

Best Practices

Best Practices Architecture14

Network Design Redundancy provides continuity for key processes

DC2 DC3

Within region: Out of region:

…....DataMirror

DataMirror

DataMirror

WS3WS2WS1 …....

Redundant technology

DC1



SOLUTIONSSOLUTIONS

today’stoday’s

Business Impact Analysis

Business Impact

Justification

Architecture & Feasibility

Implementation Choices

Cost Estimates

Recovery Solution Design & Funding

15



SOLUTIONSSOLUTIONS

today’stoday’s


Determine RTO & RPO - Design to provide varying RTO’s by service category

16

Stakeholders: Impacts:

Payors Cost of resumption, overpayment of claims

Brokers Lost sales

Employers Add to costs, dissatisfaction

Providers Slower receipt of payments

Insureds Slower receipt of claims

Step 1: Determine the needs of the business



SOLUTIONSSOLUTIONS

today’stoday’s


Best Practice vs. Traditional Compromise

Dedicated Facilities & Infrastructure vs. Shared

Mirrored / Load Balanced Sites vs. separate Production & Backup sites

Self Healing Network vs. Dial Back Up

Multiple Offices Capable of Backing Up One Another vs. Centralized Office

Real time data backup vs. daily incrementals

Step 2: Define Architecture Options

Determine Benefit Cost Tradeoffs & if affordable, adopt best practice

17



SOLUTIONSSOLUTIONS

today’stoday’s


18Sourcing Alternatives

Source:In-house Outsource Off-shore

Elements:• Mainframes X• Servers X• Network X X• Offices/Call Centers X X• Client workstations X• Redundant Sites X• Data mirroring X X

Step 3: Look at Implementation Alternatives



SOLUTIONSSOLUTIONS

today’stoday’s


Determine Costs for RTO’s19

1 day 3 days 14 days

Mainframe

• Server infrastructure

• Server data synch

• Network

• Call Center ______ ______________

TOTAL RTO COST ($):

COST OF OUTAGE ($):

Step 4: Summarize RTO Cost by Service Line and Compare to Cost of Outage



SOLUTIONSSOLUTIONS

today’stoday’s

Case Illustration:

20Representative Costs of moving toward Best Practices Scenario

IT Traditional NearBudget Recovery Best Practices

$30,000 Entprs Site $500 Hot Site $500 Hot Site 450 Vault

Data 150 Servers 200 Mirror

Data 50 Work

Area_______ __________ __________$30,000 $500 <2% $1,350 5%

Exposures: RTO Not perfect shared

RPO shared risk for MF

All costs in 000’s,exclude staff




SOLUTIONSSOLUTIONS

today’stoday’s

What Should You Do?

21Implications of changing Business Model

First steps:

Identify elements of the Business Model that are likely to change due to enterprise direction

Determine & evaluate implications of business model changes for Business Impact Analysis



SOLUTIONSSOLUTIONS

today’stoday’s

What Should You Do?

22Understand Options for Implementation

I

Adding Recovery Functions

Better Architecture

RPO

RTO



SOLUTIONSSOLUTIONS

today’stoday’s

What Should You Do?

23Balance Needs/Costs of Implementation Considerations

Adequacy of ExistingDR Plan

$ Cost of ExistingDR Plan

Current/FutureImplementation

Needs

$ Cost of Current/Future

Implementations



SOLUTIONSSOLUTIONS

today’stoday’s

What Should You Do?

24Worksteps toward a Best Practices Vision

Identify Business Model elements likely to change

Determine & evaluate implications of business model changes

DC HotSite

WS1 WS2

WS2

DC1 DC2 DC3…....

WS3WS1

RedundantTechnology

WS2

Implement justifiable increments for key business lines



SOLUTIONSSOLUTIONS

today’stoday’s

Planning for the Unthinkable…

Questions/Discussion

25



SOLUTIONSSOLUTIONS

today’stoday’s

It was a pleasure discussing Disaster Recovery & Business Continuity with you today

Dave AdolphsonManagement ChampionObjective-AIM: Automation/Innovation/Management

483 East Oxford RoadNorth Barrington, IL 60010(847) 381-1516

[email protected]

John SchladweilerInnovation ChampionSchladweiler Associates, Inc.

1630 Sheridan Road, Suite 8EWilmette, IL 60091(847) [email protected]

26

Planning for the Unthinkable…Balancing Process and Technology to Drive Value

Objective – Automation Innovation Management



SOLUTIONSSOLUTIONS

today’stoday’s

Speaker biographies

Dave Adolphson: a business-oriented information technology consultant and executive, Dave has focused his career on transforming enterprises through linking information technology, with enterprise objectives. He has a strong track record of reducing total costs, delivering to demanding schedules and budgets, and achieving tangible benefits. Currently a principal of Objective-AIM, Dave has Big Five consulting experience with KPMG, and he has also held senior IT executive roles with Aon, C N A Financial, and Harris Bank. He has spoken and written on related business and information technology topics such as acquiring and retaining profitable customers, justifying investments in information technology, and extending the application of methodology to Expert Systems and web-based implementation projects.

John Schladweiler: with 35 years of experience in the IT industry and a blend of vendor, client and consulting experience, he also brings a strong financial background to the assessment of business operations, including his understanding of the economics of existing and emerging technologies and how they can positively impact business opportunities and costs. His experience includes executive management roles with debis IT Services N. A., Realtors Information Network, and Harris Bank as well as engagements with financial services companies including The Equitable, Central Reserve, Chase Bank, and the Board of Trade Clearing Corporation, and also with leading and emerging technology companies, ASPs, and start-up eCommerce and application integration software companies. He serves on the Executive Committee of the Information Systems Management Forum, an organization focused on information exchange among CIO’s.

27

CEO Counselors does research on the relationship a chief executive has with his or her direct reports. And, we provide a process that a CEO can use to understand the root cause of what is happening. Our current research project explores the value that a business gets from investments in Information Systems and Technology. A copy of the Survey Report is available at www.ceocounselors.com.

Planning for the Unthinkable…Balancing Process and Technology to Drive Value

Objective – Automation Innovation Management

i nnovations tomorrow’ssolutions today’s hiaa forum 2003 this presentation was given at the aahp...

Documents