tasscc annual conference 2010

1

TASSCC Annual Conference 2010

Business Resiliency Business Resiliency

PlanningPlanning-Business Continuity Management--Business Continuity Management-

William Tompkins, CISSP, CBCP

Teacher Retirement System of Texas

August 2, 2010

2

William Tompkins is Business Continuity/Disaster Recovery Coordinator and William Tompkins is Business Continuity/Disaster Recovery Coordinator and Information Security Officer at Teacher Retirement System of Texas. He Information Security Officer at Teacher Retirement System of Texas. He has more than 26 years of technical, managerial and consulting experience has more than 26 years of technical, managerial and consulting experience in information technology and more than 18 years in business continuity in information technology and more than 18 years in business continuity and information security planning. He is a Certified Business Continuity and information security planning. He is a Certified Business Continuity Professional and a Certified Information Systems Security Professional. Professional and a Certified Information Systems Security Professional.

He is the current President of the Association of Contingency Planners chapter He is the current President of the Association of Contingency Planners chapter in Austin.in Austin.

William was elected to the ISSA Hall of Fame in 2006 by the ISSA William was elected to the ISSA Hall of Fame in 2006 by the ISSA International Board of Directors. (International Board of Directors. (IInformation nformation SSystems ystems SSecurity ecurity AAssociationssociation))

Mr. Tompkins holds two Bachelor of Science degrees, Psychology and Mr. Tompkins holds two Bachelor of Science degrees, Psychology and Computer Information Science, from Troy State University in Alabama and Computer Information Science, from Troy State University in Alabama and Certification in Risk Management from University of Texas at Austin Certification in Risk Management from University of Texas at Austin Division of Continuing Education.Division of Continuing Education.

William TompkinsWilliam Tompkins

3

In this session we’ll overview In this session we’ll overview business resiliency practices business resiliency practices at at Teacher Retirement System of Texas, including , including our planning & maintenance our planning & maintenance practices, coordination with practices, coordination with other agencies, business other agencies, business partners, and our contracted partners, and our contracted recovery service provider.recovery service provider.

4

AgendaAgenda

Why ?Why ?

How ?How ?

What ?What ?

Q & AQ & A

6

Are we ready?

PresumptionsPresumptions

RealityRealityRealityReality

*from Managing Managers: A Case Study by Philip Jan Rothstein; Copyright 1995, Rothstein Associates Inc.

Versus

7

PresumptionPresumption

RealityReality

The “wizards” (in IT Div.) could handle any crisis and business would be operational within a few hours.

At best, recovery from a MAJOR disruption could take 30-36 hours.


Are we ready?Are we ready?

8

RealityReality

IT Div. “automatically” integrates all diverse technology and platforms into the Disaster Recovery Program.

IT Div. did not implement -OR- operate many of these platforms.




9

RealityReality

No matter what the cause or scope of disruption ... IT Div. would recover all data accurately AND to the point of failure.

At best, recovery would be to the prior night's backup and, most probably, to a point at least 3 to 4 nights prior.




10

RealityReality

Data entry sections have manually filed source documents, so the data entered since the last backup is clearly identified.

?

?



11

RealityReality

Well-respected practitioner had a very good program.

Senior management was dissatisfied with the program…because the organization’s professionals were not familiar with the real business processes.



12

TimelineTimelineTimelineTimelineReturn to Normal

Operations

Evaluate&

Decision

Restore(most data &

some infrastructure)

Recovery(weeks to months?)

Begin ResponseMobilize people &

Notify recoverycontractors

Min. 4 hrs.Max. 12 hrs.

12 – 24hours

48 hours Staff begins re-entering Tues work no earlier than Saturday morning

72 hrs72 hrs

13

By the end of this session . . . better understanding of business resiliency

Administrative activitiesAdministrative activities

Planning activitiesPlanning activities

Technical activitiesTechnical activities

User educationUser education

14


Policy Policy (Business Continuity Management Policy)DefinitionsDefinitions

Business Impact Assessment (BIA)Business Impact Assessment (BIA)Mission criticalMission critical

RolesRolesBusiness Continuity/Disaster Recovery Business Continuity/Disaster Recovery

CoordinatorCoordinator -vs- Business Continuity Planner -vs- Business Continuity Planner

15

RolesManagement support:Management support:

Executive management Executive management Project initiation, scope, final approval, ongoing supportProject initiation, scope, final approval, ongoing support

Senior business unit management Senior business unit management Identifies and prioritizes time-critical systemsIdentifies and prioritizes time-critical systems

Functional business units Functional business units (departments)

Participate in implementing and testingParticipate in implementing and testing

16


Reporting Reporting Annual BCP Annual BCP – a summary report – a summary report

includes copy of up-to-date BIA and dates includes copy of up-to-date BIA and dates of IMT Plan, Incident Response Plan, of IMT Plan, Incident Response Plan, business unit continuity plans business unit continuity plans & IT’s DR & Telecommunications plans& IT’s DR & Telecommunications plans

After-action of Hot Site ExerciseAfter-action of Hot Site ExerciseResults of “primary” & “secondary” Results of “primary” & “secondary”

objectivesobjectivesAnnual Risk Assessment

17

Program Goal( from “Policy” )

……to prepare to counteract interruptions to prepare to counteract interruptions

to TRS’ business activities and to to TRS’ business activities and to

protect critical business processes protect critical business processes

from the effects of disasters or major from the effects of disasters or major

failures of information systems and to failures of information systems and to

ensure their timely resumptionensure their timely resumption

18

PlanningPlanning

19

What is BIAWhat is BIA A Business Impact Analysis (‘BIA’) identifies and A Business Impact Analysis (‘BIA’) identifies and

prioritizes the critical business processes supported prioritizes the critical business processes supported by the technology infrastructure. by the technology infrastructure.

BIA Key Components:BIA Key Components:

Identifies the impact of potential resource lossIdentifies the impact of potential resource loss

Identifies the minimum resources needed to recoverIdentifies the minimum resources needed to recover

Prioritizes the recovery of processes and supporting Prioritizes the recovery of processes and supporting systemssystems

Establishes the escalation of that loss over timeEstablishes the escalation of that loss over time

20

Impact priority considerationsImpact priority considerations{not in priority sequence}

★Required by law

★Critical or essential business need

★Inaction (or incorrect action) violates fiduciary duty

★Inaction causes harm

★Impacts large number of people

★Severe adverse impact on TRS’ mission, functions, or reputation

21

BIA QuestionsBIA QuestionsWhat are the critical functions?

Why are they critical?How quickly does it need to be recovered? Why?Does it need to be recovered in the event of a disruption/disaster?

If it is not recovered as quickly as it needs to be, what will happen? So what? Who else would be affected?

22

Chart legend for following pages

25

Contingency PlanningContingency Planning

Risk Management identifies risks that Risk Management identifies risks that require contingency plansrequire contingency plans

Risk decisions are based on BIA detailsRisk decisions are based on BIA details

Contingency plans - business decisions Contingency plans - business decisions based on real numbers and facts.based on real numbers and facts.

26

Contingency Plans Are:Contingency Plans Are:

Interim recovery measuresInterim recovery measures that ensure survival of the that ensure survival of the organization during a disaster event by providing for organization during a disaster event by providing for continuity of its critical business functions.continuity of its critical business functions.

Long term outage provisions Long term outage provisions

Critical system relocation proceduresCritical system relocation procedures

Personnel issues Personnel issues –– get the right people to right place get the right people to right place

(Internal) Temporary business operation modes (Internal) Temporary business operation modes

(External) How to deal with customers, partners, and (External) How to deal with customers, partners, and shareholders through different channelsshareholders through different channels

27

Planning activitiesPlanning activities

Business Continuity Plan ?Business Continuity Plan ?

Not exactly . . .Not exactly . . .

Business Resiliency Program ?Business Resiliency Program ?

YesYes

28

PlansIncident Management Team PlanIncident Management Team Plan

Single reference for Exec & Sr. MgmtSingle reference for Exec & Sr. Mgmt

Crisis Management Plan*Crisis Management Plan*After initial ‘triage’ …helps clarify eventAfter initial ‘triage’ …helps clarify event

What happenedWhat happened

How seriousHow serious

What to do nextWhat to do next

Incident Response Plan*Incident Response Plan*Addresses initial stages of any eventAddresses initial stages of any event

*Note: in some organizations, crisis management & incident response is the same*Note: in some organizations, crisis management & incident response is the same

29

Disaster Recovery PlansDisaster Recovery Plans

Enable quickly resuming operations Enable quickly resuming operations for most critical unitsfor most critical units

Network infrastructureNetwork infrastructure

IBM Mainframe IBM Mainframe (business class enterprise server)

& a midrange system& a midrange system

Telecommunications Telecommunications

Plans

30

PlansBusiness Continuity PlansBusiness Continuity Plans

Covers all critical and major business units (17), Covers all critical and major business units (17), provides detail for staff involved in early recovery effortsprovides detail for staff involved in early recovery efforts

Business UnitsGeneral Accounting External Public Markets Insurance (TRS Care)

Benefit Accounting Private Markets Insurance (Active Care)

Benefit Processing Trade Management General Counsel

Benefit Counseling Internal Public Markets Mail & Supplies Center

Member Data Services Investments (Research & Risk) Printing & Bindery

31

Site Restoration PlanSite Restoration Plan

Plan for restoration efforts by facilities Plan for restoration efforts by facilities

Vital Records Retention / Recovery Vital Records Retention / Recovery PlanPlan

Change Management PlanChange Management Plan

Include plan updates when technology or Include plan updates when technology or business process changesbusiness process changes

Plans

32

Technical Technical ActivitiesActivities

33

Data Back Up Data Back Up Routine BackupsRoutine Backups

Retention Retention Daily – 14 daysDaily – 14 days

Weekly – 6 monthsWeekly – 6 months

Monthly – 2 monthsMonthly – 2 months

Archive Archive [EOY] [EOY] (annual: Member data, based on (annual: Member data, based on annuitant)annuitant)

Mainframe{incl. Imaging (Filenet)}

Network

Nightly: All PROD data (approx. 6 hours) Weekly: Friday night (approx. 18 hrs)

Weekly: also incl. Devl. & Test data (11 hours) Mon. – Thu.: incremental backups

Monthly: same as weekly Monthly : “last Friday” copy

34

Actual “Hot-site” exerciseActual “Hot-site” exercise

Investment Div. has been actively Investment Div. has been actively involved in at least 6 exercisesinvolved in at least 6 exercises

Emergency Call Tree Exercise Emergency Call Tree Exercise

Tabletop exerciseTabletop exercisea sit down desk-check with the team a sit down desk-check with the team

leaders and team membersleaders and team members

Testing & ValidationTesting & Validation

35

User User EducationEducation

36

User AwarenessUser Awareness

Providing Awareness, leads to…

Understanding

Change in AttitudeChange in Attitude

Change in Behavior!Change in Behavior!

37

Executives and Senior Managers/DirectorsExecutives and Senior Managers/Directors

ongoing familiarization sessions w / Sr. Mngt.ongoing familiarization sessions w / Sr. Mngt.

Business Unit Managers & Team LeadersBusiness Unit Managers & Team Leaders

considerations as business processes and business considerations as business processes and business

partners changepartners change

Regular Staff, Temp Hires & ContractorsRegular Staff, Temp Hires & Contractors

introductory classes given to new employeesintroductory classes given to new employees

User AwarenessUser Awareness

38

External External “Partners”

39

PrivatePrivate““Hot-site” contractHot-site” contract

Business Continuity & Resiliency ServicesBusiness Continuity & Resiliency Services

Off site storageOff site storage Tape backups & hardcopiesTape backups & hardcopies

Other State AgenciesOther State Agencies TCEQ TCEQ – Backup Command Center– Backup Command Center TPASSTPASS ((Tx Procurement & Support SvcsTx Procurement & Support Svcs) – Mail ) – Mail TxDOTTxDOT – Web site– Web site

External Contracts & MOUsExternal Contracts & MOUs

40

Self-assessmentSelf-assessment

41

www.theiia.org/technology

www.theiia.org/technology

42

Q U E S T I O N S ?Q U E S T I O N S ?Q U E S T I O N S ?Q U E S T I O N S ?

Thank YouThank YouThank YouThank YouWilliam A. TompkinsWilliam A. Tompkins

(512) 542-6787(512) 542-6787

[email protected]@trs.state.tx.us

William A. TompkinsWilliam A. Tompkins(512) 542-6787(512) 542-6787

[email protected]@trs.state.tx.us

tasscc annual conference 2010

Documents