Application Sustainability Assessment Framework within the

Canada Revenue AgencyDavid Gibson, Solutions Quality ProgramInformation Technology BranchCanada Revenue Agency

GTEC 2008October 29, 2008

What You Can Expect from this Presentation

• An overview of the application sustainability framework (ASAF) process within CRA

• Challenges, opportunities and lessons learned • Access to the ASAF documentation and

framework

2

Agenda

• Introduction • Application Sustainability Overview• Application Sustainability Assessment

Framework• Application Sustainability Process• Application Report Cards• Questions / Contact Information

3

Introduction

History: In September 2006, CRA’s Information Technology Branch (ITB) participated in the Integrated Risk Management Program’s - Risk Identification and Assessment workshop

Major Conclusion: Sustainability of aging applications was among the highest risks facing CRA for the future.

Result: Application Sustainability Assessment project was initiated to address this risk, by developing a framework (ASAF) and a continuous improvement process to measure and manage sustainability. 4

CRA Corporate ProfileMission:

To administer tax, benefits and related programs to ensure compliance on behalf of governments across Canada, thereby contributing to the ongoing economic and social well-being of Canadians

Size of Organization: 40,000 full-time employees, 3000 in ITB.

Application Portfolio: 454 corporate applications, 35 million lines of code.

Annual Statistics:• Collection: $350 billion annually, averaging $1.3 billion per working

day• Tax Return Processing: 25 million individual; 1.6 million business,

6.8 million GST/HST• $15 billion benefit payments distributed to 11 million Canadians• CRA Web-site: 36 million visits; 61 million publications

downloaded• Public Enquiries: > 25 million enquiries answered 5

Application Sustainability OverviewSoftware is an Asset and like all assets, their value needs to be amortized and depreciated with plans made for maintenance, re-factoring and eventual retirement as they deteriorate normally.

Some Facts:

• Large systems are delivered with thousands of latent bugs• Roughly 7% of all defect repairs will introduce a new defect• Defects are not randomly distributed and patterns can be

identified• Continuous growth and software aging increases complexity• Software entropy (tendency to destabilize) is natural and

predictable

Source: Quality Assurance Institute Worldwide

ASAF provides one tool for understanding asset degradation, and total cost of ownership by introducing a framework for measurement, and a process for management of the software.

6

Application Sustainability Overview• As applications continue to age beyond the 8-9 year point,

productivity begins to decline 2-3% per year • And up to the 8-9 year mark, productivity increases

System Age to Support Productivity Relationship

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

3000

0 2 4 6 8 10 12

Age of Applications (Years)

Fu

nc

tio

n P

oin

ts S

up

po

rte

d p

er

Pe

rso

n

CRA’s Portfolio

Average Age

7

Defining Sustainability

“Sustainability”: the ability of an application to continue to meet requirements within accepted cost and quality constraints.

To “Assess” sustainability, we must:• understand application requirements

• measure application performance

• know acceptable limits of cost and quality

• have a well-defined vision for the future

“The Goal”: to know where we are, where we are going, and what the road ahead will look like.

8

Application Sustainability Assessment Framework

Operations

• Technology alignment• Application complexity• Architectural alignment

Architecture• Consult business clients

to determine development plans and priorities

• Business/IT alignment

Business

• Annual maintenance and support costs• Application Quality• Availability of support skills• Compliance with operational policies and processes

Operations

9

Business Constraint

Business requirements evolve continuously over time to address:

• Changing market conditions• Changing client needs • Legislative and Regulatory changes

Impacts to Sustainability:• Increased complexity and difficulty supporting new

requirements based on existing design;• Higher maintenance costs• Increased change request turn-around time• Decrease in application quality

10

Architecture Constraint• Evolution of technology can occur at a rapid pace, driven

by improvements in hardware, software, database and programming languages

• Technologies also go through a natural lifecycle from infancy to elderly, effecting the sustainability of applications dependant on them.

Impacts to Sustainability:• Lack of vendor support• Diminishing community and 3rd party markets• Fewer available skilled resources• Declining compatibility and interoperability

11

Operations Constraint

Provides the day-to-day view of people, processes and policies relating to the applications operating environment.

Encompasses a wide variety of areas, including workforce availability, skill sets, organizational process and procedures, change and problem management

Impacts to Sustainability:• Budget Inadequacy• Unavailability of required skill sets as workforce ages• Non-compliance with established SDLC procedures

and best practices

12

The ASAF Process

The ASAF uses a continuous improvement process based on the spiral model as the basis for its’ yearly assessments.

The approach has 5 steps.

Step 1: Measure

Step 2: Analyze

Step 3: Communicate

Step 4: Manage

Step 5: Improve

13

Step 1: MeasureSustainability assessment results are based on the measurement of available metrics. It is expected that over time the number of metrics, and collection methods will increase.

Where does the Source Data come From?CRA Measurements and Metrics programs provide the source data collected for use by the ASAF, including: CRA Configuration Management;

• Change / Problem Management: INFOMAN, Remedy• The Solutions Application Catalogue: SAC• Client, Architect and Application Owner Questionnaires

Technical Standards• Application / Technology Framework• Technology Architecture Outlook

14

Step 1: Measure (cont.)

Lessons Learned• Initially, measure what is available• Collect only measurement that will be used• Increase the number of metrics gradually over time• Preview Questionnaires / Surveys with the clients

Methods of Collection• Web Applications (through the SAC)• Questionnaires• Automated Interfaces and Reports• Metrics

15

Step 2: AnalyzeMeasurement quantifies the software asset for each of the metrics collected, then compares using a standardized rating scale

ASAF scoring algorithm is applied to create an overall sustainability score which is used for comparison and evaluation of score distributions to identify trends and patterns.

Distribution results are compared by: • individual metric;• by individual application, • by line of business, organizational structure, etc.

Lessons Learned

Adaptable to any organization because scoring algorithm can be aligned to available metrics and enterprise goals by weighting individual factors

The scoring algorithm does not have to be complicated to identify sustainability patterns and trends 16

Result: The Balanced View

The result is an assessment value by constraint (per application)

The combining of these values results in a single Balanced View

17

The ASAF Sustainability Matrix

The ASAF Sustainability Matrix is based on the TBS – Integrated Risk Management Framework

Provides a view of:• Sustainability vs. Business Priority

PO

OR

Significant management

required: consider architectural and

business transformation

Significant management

required: consider business

transformation and architectural fine-

tuning

Extensive management

essential: consider architectural and

business transformation

Considerable management

required: consider application retirement and architectural and

business transformation

Moderate management

required: consider business

transformation and architectural fine-

tuning

Moderate management

required: actively monitor sustainability

and business value

Moderate management

required: actively monitor sustainability

and business value

Moderate management

required: consider application retirement

business transformation, and architectural fine-

tuning

Insignificant management

required: consider application retirement

and business transformation

GO

OD

FA

IRLOW MEDIUM HIGH

SU

ST

AIN

AB

ILIT

Y

BUSINESS PRIORITY 18

Other views of Application SustainabilityMany organizations are now promoting application sustainability.

19

Application Report CardsApplication Report cards provide a snapshot of an individual applications overall health

• Summary

• Architecture

• Business

• Operations

20

Report Cards: Summary

A one page summary for each application, indicating the current and previous assessments Balanced View, Constraints and general trends (increase / decrease)

21

Report Cards: Architecture

Provides a summary of the Architecture metrics and score results for the application

22

Report Cards: Business

Provides a summary of the Business metrics and score results for the application

23

Report Cards: Operations

Provides a summary of the Operations metrics and score results for the application

24

Overall Assessment Results: Aggregated

ASAF Score Distribution Summary Reports (produced yearly)

25

Overall Assessment Results: Individual

Reporting by Individual ASAF Scorecard Metrics (produced yearly)26

Overall Assessment Results: Matrices

Examples of ASAF Reports By: • Balanced View• Line of Business• Technology Alignment to Enterprise Architecture.

27

Overall Assessment Results: Line of Business

Reporting by Line of Business (produced yearly)28

Forecasting the Balanced View

Plotting of the yearly assessments of the Balanced View allows the ability to forecast sustainability, and compare results of previous action plans 29

Step 3, 4, & 5: Communicate / Manage / Improve

ASAF provides for the identification and forecasting of sustainability patterns and trends, but there is no silver bullet.

Communication is established through bilateral presentations and meetings, with the goal of highlighting sustainability risks and determining a course of action.

ASAF results can be used as input to:• Planning & prioritization exercises• Alignment to the enterprise architecture where required• Preparation of architectural transition options • Preparation of a business case for funding

30

Desired Outcomes of Measuring Sustainability

31

Future Sustainability Activities at CRA

ASAF • Assess CRA Local Application Repository• Continued enhancements to the ASAF web

application as part of continuous improvement process

• Providing input for major business cases to modernize applications

SAC• Continued integration of the SAC and ASAF

Metrics• Gradual increase in the number of Metrics collected

and used by ASAF

Measurements• Continued automation of measurements interfaces

for collection of data 32

Conclusions

• Software is an Asset: Sustainability helps plan for Maintenance, Refactoring and eventual Retirement.

• Sustainability Assessments highlight predictable patterns that can be used for planning, prioritization and reduction of the total cost of ownership of applications.

• The ASAF provides a customizable framework and process that can be adapted quickly and easily by any organization.

33

Questions

34

For More Information about the ASAF….

Please contact:David GibsonSolutions Quality Program DivisionInformation Technology Branch4-29 875 Heron Road(613) 941-8542David.Gibson@cra-arc.gc.ca

35